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(57) Abstract: A multiple camera 
video system and methods for 
operating such a system. The 
system may include a plurality of 
cameras located around a stadium, 
athletic playing field or other 
location. The cameras are remotely 
controlled in a master- slave 
configuration. A camera operator 
at a master pan head selects one 
of the plurality of cameras as the 
current master camera and utilizes 
the master pan head to adjust the 
telemetry and zoom of the master 
camera to follow the target object 
The telemetry and zoom parameters 
of the master camera are then 
used to calculate corresponding 
telemetry, zoom and/or other 
parameters for each of the plurality 
of slave cameras. Video captured 
by each of the cameras is stored for 
the production of replay video feeds 
or for archiving. The replays may 
be capable of "spinning" through 
the video feeds of adjacent cameras 
in order for the viewer to get the 
sensation of revolving around the 
target object The multiple camera 
video system also includes methods 
for calibrating the system. 



WO 03/041411 Al lllilM 



— before the expiration of the time limit for amending the For two-letter codes and other abbreviations, refer to the "Guid- 
claims and to be republished in the event of receipt of ance Notes on Codes and Abbreviations" appearing at the begin- 
amendments ning of each regular issue of the PCT Gazette. 



WO 03/041411 



PCT/US01/46357 



VIDEO SYSTEM AND METHODS FOR OPERATING A VIDEO 

SYSTEM 



BACKGROUND OF THE INVENTION 

FIELD OF THE INVENTION 

The present invention relates to video systems and 
methods of operating such systems, and more 
particularly, the invention relates to video systems 
employing a plurality of cameras to produce images of 
a target object from various spatial perspectives. 

DESCRIPTION OF THE BACKGROUND 

In the televised broadcast of live events, it is 
frequently desirable to replay portions of the events 
for further analysis and/or to enhance the viewing 
experience. One example, the familiar instant replay 
feature, is used in televised sporting events to 
review sports plays. By replaying video taken by 
cameras at different locations, it may be possible to 
obtain a better view of the event than was provided by 
the camera selected for the original broadcast. 

Multiple camera imaging has been the subject of 
several issued patents. For example, U.S. Patents No. 
5,729,471 and 5,745,126 disclose a multiple, camera 
television system in which an object of interest 
(target object) can be viewed from multiple spatial 
perspectives using cameras that provide different 
views of a scene. 

However, such multiple camera television systems have 
typically utilized fixed position cameras. In the 
telecast of live events, target objects are often in 
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motion, and it is desirable to be able to move the 
camera to follow the movements of particular objects. 
Furthermore, when replaying video clips of events such 
as sporting events, it may be further desirable to be 
able to view the events from multiple spatial 
perspectives. 



SUMMARY OF THE INVENTION 

[5] In at least one preferred embodiment, the present 

invention provides a camera system and method of 
operation of the camera system that uses multiple 
cameras to produce video images from multiple spatial 
perspectives. The system may also permit the replay 
of those images to view an event from different 
spatial objectives. 

[6] The system preferably includes a plurality of video or 

other cameras located at various spatial locations 
around a target object, such as a sports stadium or 
athletic field. Each of the cameras may include a 
robotic mount allowing for the remote control of the 
positioning, zoom, focus, and/ or other aspects of the 
camera. One camera is selected as the current master 
camera, and the remaining plurality of cameras are 
robotically controlled as slaves to the master camera 
which follow the master camera's field of view from 
different spatial perspectives. 

[7] The plurality of cameras may be controlled from a 

remote location which includes a master pan head used 
to directly control the selected "master" camera. As 
the master pan head is moved and adjusted as reflected 
in a local video monitor, a master broadcaster 
computer sends information (collectively, "telemetry") 
to the actual master camera by way of a fiber optic, 
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tri- axial, wireless RF, . or other communications 
connection. The master broadcaster computer also uses 
a software program to calculate new positional 
coordinates and camera settings for each of the 
plurality of "slave" cameras so that each of the 
plurality of cameras is capturing an image of the 
target object as defined by the camera operator at the 
remote master pan head. 

[8] The video signal captured by each of the plurality of 

cameras is preferably stored in a plurality of digital 
disc recorders (DDRs) , a file server or other storage 
device. The DDRs may be connected to a digital router 
capable of outputting the content of any of the DDRs. 
A slow motion controller connected to the DDRs and/or 
the router allows a broadcast engineer to move forward 
and backward at varying speeds through the stored 
content as well as to select which of the plurality of 
cameras is to be output from the router. By selecting 
to output the video feed from adjacent cameras in 
quick succession, the output image gives the sensation 
of "spinning" or "rotating" around the target object. 
The selected series of images from the plurality of 
cameras can then be broadcast directly to air or can 
be re-recorded for further editing or pre-production 
with the aid of an additional slow motion controller 
and storage device. 

[9] The system may also include one or more calibration . 

stations capable of calibrating the positioning of the 
multiple cameras used to capture the video images. 
The calibration station preferably includes a software 
program running on at least one calibration computer 
that allows dynamic calibration of the positional 
coordinates and other attributes for each of the 
plurality of cameras. 
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[10] The present invention may also include a zoom 
calibration program that allows the multiple slave 
cameras to adjust the zoom size (field of view) based 
on the positional coordinates of the target object and 
the zoom size of the master camera. This zoom 
calibration allows the target object to appear as the 
same size in the video frames from each of the cameras 
when the video output is rotated through adjacent 
cameras. Likewise, there may also be a focus table 
and focus calibration program. 

[11] There may also be one or more paint stations including 
computer programs capable of adjusting the iris level, 
color settings, shutter speed, and other attributes of 
each of the plurality of cameras. This program may 
include the functionality to simultaneously change the 
attributes of all or a selected group of the cameras . 
These results are preferably communicated to the 
cameras by way of the master computer. 



BRIEF DESCRIPTION OF THE DRAWINGS 

[12] For the present invention to be clearly understood and 
readily practiced, the present invention will be 
described in conjunction with the following figures, 
wherein like reference characters designate the same 
or similar elements, which figures are incorporated 
into and constitute a part of the specification, 
wherein: 

[13] Figure 1 is a top level block diagram of the 

components of a multiple camera video system; 

[14] Figure 2 is a schematic representation of an 

exemplary camera system; 
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[15] Figure 3 is a schematic representation of 

exemplary camera control attributes; 

[16] Figure 4 is a schematic representation of a 

master camera following a target object across a field 
of play; 

[17] Figure 5 is a schematic representation of the 

geometric relationships between a camera and a target 
object; 

[18] Figure 6 (6a-6e) shows simulated frames of 

video images that illustrate the operation of the 
present invention in a video replay mode; 

[19] Figure 7 (7a-7e) shows simulated frames of 

video images that further illustrate the operation of 
the present invention; 

[20] Figure 8 is a system diagram of the camera 

system and the camera control system; 

[21] Figure 9 shows an exemplary robotic pan head; 

[22] Figure 10 is a schematic view of one exemplary 

embodiment of the wiring inside the camera controller 
panel ; 

[23] Figure 11 is a schematic of one exemplary 

embodiment of a camera control system; 

[24] Figure 12 shows a sample calibration computer 

program screen shot; 

[25] Figure 13 shows an exemplary data file with 

absolute positional coordinates (raw data) ; 

[26] Figure 14 shows an exemplary data file for a 

camera with calibration, zoom, and focus tables; 
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[27] Figure 15 shows an exemplary screen capture 

from a paint station computer program display; 

[28] Figure 16 shows an exemplary DDR-based image 

storage and playback system; 

[29] Figure 17 shows a schematic view of a football 

field with a plurality of spaced microphones; and 

[30] Figure 18 shows an exemplary use of "virtual 

cameras" between actual cameras at a football game. 



DETAILED DESCRIPTION OF THE INVENTION 

[31] It is to be understood that the figures and 
descriptions of the present invention have been 
simplified to illustrate elements that are relevant 
for a clear understanding of the invention, while 
eliminating, for purposes- of clarity, other elements 
that may be well known. Those of ordinary skill in 
the art will recognize that other elements may be 
desirable and/or required in order to implement the 
present invention. However, because such elements are 
well known in the art, and because they do not 
facilitate a better understanding of the present 
invention, a discussion of such elements is not 
provided herein. The detailed description will be 
.provided hereinbelow with reference to the attached 
drawings . 

[32] The present invention contemplates, in at least one 
preferred embodiment, a system and method for a 
multiple camera video system. The system remotely 
controls a plurality of cameras in a master/slave 
relationship to capture images of a target ■ object from 
multiple spatial perspectives. These multiple video 
feeds may thereafter be played back from a storage 
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device and further edited to produce replays with the 
functionality to rotate around the target object as 
the object is in motion. 

[33] A brief overview of the system will first be given 
from a high level of abstraction. Thereafter, a more 
detailed discussion of the overall system as well as 
each of the system components will be described. 
Finally, examples of system use will be provided to 
more specifically point out how the system components 
work together. 

[34] Referring to the drawings, PIG. 1 is a block diagram 
of the general system components of the present 
invention. The FIG. 1 multiple camera video system 1 
includes a robotic camera system 10 for capturing 
video of a moving target object from a plurality of 
spatial perspectives, an image storage and playback 
system 3 for recording the captured images and 
producing replay video feeds, and a camera control 
system 5 for calibrating and remotely controlling each 
of the cameras of the robotic camera system 10. The 
camera system 10 may be communicatively coupled to the 
other systems 3, 5 by way of a fiber optic, triaxial, 
RF wireless, or other medium 7. The camera control 
system 5 and the image storage and playback system 3 
may exist at the same location, for example in a 
broadcast truck, or various parts of either system 3, 
5 may be located remotely from other system 
components . 

[35] The cameras 10a are preferably mounted on robotic pan 
heads 10b that allow for remote control of the 
positioning (pan, tilt) , zoom and/or other camera 
attributes. Each camera 10a also has a camera 
controller panel 10c associated with the camera that 
provides power and communications capabilities to the 
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cameras 10a. Positioning signals are brought up to 
the cameras 10a, and the captured video feed is sent 
back down to the components of the camera control 5 
and the image storage and playback systems 3 . The 
following description refers generally to video 
cameras, however, it should be understood that such 
cameras could be standard resolution cameras, high 
definition video cameras, and/or other devices capable 
of capturing video image signals. 

[3 6] The camera control system 5 includes the components 
used for calibrating, positioning, and/or adjusting 
various attributes of the cameras 10a. The camera 
control system 5 preferably comprises both 
communications elements (e.gr., Ethernet media and 
fiber optic video converters) and control elements. 
There may be a master pan head 5a with a mounted 
monitor that allows a camera operator to remotely 
control a selected "master" camera by moving the 
master pan head 5a. There may also be a master 
broadcaster computer 5b capable of calculating the 
correct pan, tilt, zoom, and/or other camera 
attributes for the master and slave cameras and 
communicating these values up to each camera 10a. 
There may be a cam- A computer 5c that provides visual 
feedback and functionality about the master camera. 
Additionally, the camera control system 5 preferably 
comprises one or more calibration stations 5d and 
paint stations 5e to set up and adjust the various 
attributes of the cameras 10a. 

[37] The image storage and playback system 3 includes the 
components needed to store the captured video from 
each of the plurality of cameras 10a as well as the 
functionality to produce instant replays and other 
video feeds from this captured video. An operator of 
the image storage and playback system 3 is able to 
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produce a video segment while controlling the speed, 
camera position, and/or movement (forward or backward) 
of the stored video. This functionality may be 
implemented using one or more video storage devices 
3a, 3b (e.g., Digital Data Recorders "DDRs") and one 
or more slow motion controllers 3c, 3d. The produced 
video segment may then be broadcast to air 9 or re- 
recorded for further production. 

[3 8] FIG. 2 is a schematic representation of an exemplary 
camera system 10 constructed in accordance with at 
least one presently preferred embodiment of the 
invention as applied to a television broadcast of an 
American football game. The system includes a 
plurality of television cameras 12, 14, 16, 18, 20, 
22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 
48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68 and 70 
positioned at different locations around the field of 
play 72 . These cameras may be mounted to the stadium 
structure or other secure location, and are preferably 
raised well above the field level for the best vantage 
of the target object on the field. While this example 
shows the use of thirty cameras, it should be 
understood the invention is not limited to a 
particular number of cameras, and a greater or lesser 
number of cameras may be used within the scope of the 
invention. The cameras are located at spaced 

positions to provide video images of various target 
objects in the field of play from various spatial 
perspectives. The cameras may also include 

microphones that transmit sound along with the 
captured video. 

[39] The multiple camera system of the present invention 
generally operates by selecting and controlling one of 
the plurality of cameras, thereby making it the 
"master" camera, from a remote master pan head which 
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may be located in a broadcast truck. The camera 
operator uses the master pan head to position the 
master camera to follow a target object on the field 
of play 72. For example, the target object during a 
football game may be the football or a particular 
player on the field 72. As the camera operator 
utilizes the master pan head to select and control a 
master camera to follow the target object, information 
signals representative of the master pan head's (and 
therefore the master camera's) positional movement, 
focus and framing are sent from the pan head to the 
master broadcaster computer. This information is then 
used by the master broadcaster computer to calculate 
actual positioning coordinates and other attributes 
for the master camera. These values are also used by 
a geometric transform function in the master 
broadcaster computer to calculate positioning 
coordinates and other attributes for each of the 
remaining "slave" cameras (all cameras except for the 
current master camera) so that all cameras in the 
system will be capturing video of the same target 
object with approximately the same size field of view. 
The camera that is designated as the master camera at 
any one time is preferably selectable from any of the 
system cameras. 

[40] In order to achieve coordinated operation of all of 
the cameras, the camera control system 5 must be 
provided with information about each camera, such as 
its location with respect to the field of play (or 
collected calibration points on the field of play) , 
vertical and horizontal orientation, field of view, 
focus distance, and/or other characteristics-. FIG. 3 
is a schematic representation of an exemplary camera 
42 for use in a system constructed in accordance with 
the present invention. FIG. 3 shows the various 
camera parameters that preferably are known, such as 
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the location of the camera in 3D space (shown as 
coordinates X,Y,Z) and parameters that can be 
controlled by the associated camera control system 5 
(such as the pan direction 102, the tilt direction 
104, and/or the rotational angle 106). The pan, tilt 
and rotational angles of the individual cameras can be 
controlled by mounting the cameras on a robotic pan 
head or mount with the position of the platform being 
controlled automatically by various arrangements of 
servo motors and mechanical couplings. The focus and 
zoom functions can also be controlled remotely as 
described below. 

[41] The location of each of the cameras with respect to 
the scene to be imaged (e.g., the field of play) or 
with respect to each other may be determined using 
known techniques, such as by using a surveying 
theodolite, a global positioning system (GPS) device, 
a triangulation method or some other device/method to 
determine the location of the cameras with respect to 
* predetermined landmarks in the scene, such as the 
corners of a football field. For accuracy in the 
camera control system 5, each camera should be 
calibrated based on a series of captured calibration 
points on a computer- created texture map of the field 
of play. Such a calibration texture map may account 
for local irregularities in the field of play, such as 
the pitcher's mound on a baseball diamond. 

[42] Once the locations ,of all of the cameras are known, 
this information can be combined with other 
calibration information to create a geometric 
transform function used by the master broadcaster 
computer to repeatedly calculate settings for each 
slave camera based on information retrieved from the 
master pan head (or directly from the master camera). 
These setting are preferably calculated approximately 
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120 times per second. These settings (e.g., pan, 
tilt, zoom, focus, etc.) will be communicated by the 
master broadcaster computer to the individual camera 
controller panels to position and focus the slave 
cameras and to adjust the framing so that the object 
of interest is substantially the same size in each of 
the video images produced by the cameras. These 
transform functions are based on rudimentary geometric 
relationships between the cameras and between each 
camera and the field of play. 

[431 In at least one preferred embodiment, the master 
camera is remotely controlled by a master pan head 
used to follow a target object on or around the field 
of play. The target object should be positioned 
within a portion of the camera's field of view 
referred to as the "sweet spot." The sweet spot would 
■ typically be near the center of a camera's field of 
view, but through the use of appropriate signal 
processing, other parts of the field of view could be 
used. The framing of an object at the sweet spot may 
change in relation to the action being shot. For 
example, at the beginning of a football play, the shot 
might be wider to include most of the players, but as 
the focus is narrowed to one or two players, so would 
the framing of the master camera, and thus the slave 
cameras . 

[44] As the master camera tracks the target object, the 
slave cameras are automatically controlled to track ' 
the target object in a manner that permits subsequent 
viewing of the various images recorded by the cameras. 
That is, the automatically controlled slave cameras 
should focus on the target object and adjust the field 
of view so that the target object is substantially the 
same size in the images recorded by the various 
cameras. In the preferred embodiment, the captured 
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images would be stored in a digital disc recorder or 
file server for further production and editing, as in 
an instant replay. 

[45] The operation of the invention can be further 
described by referring to FIG. 4. In FIG. 4, assume 
that camera 12 is the currently selected master camera 
and is initially focused on a target object, such as a 
football player at location 108. By using known 
locations of the slave cameras 14-70, the distance Dl 
from camera 12 to location 108 and the framing size 
(or zoom) information for camera 12, the master 
broadcaster computer can calculate the information 
needed to direct all of the additional (slave) cameras 
14-70 to focus on the target object 108 and to adjust 
the frame size so that the target object appears to be 
substantially the same size in the video images 
produced by each of the cameras 12, .14-70. This 
calculated information is then sent to the camera 
control panels associated with each of the additional 
cameras, and the camera control panels move the 
cameras to the correct .position. 

[46] If the target object moves in a straight line from 
location 108 to location 110, the tilt angle of the 
master camera 12 will change to follow the object. If 
the field of view is not changed at the master camera 
12, the target object will appear smaller due to the 
increased distance D2 between the master camera 12 and 
the target object 110. Slave camera 42, which is 
located directly across the field from the master 
camera 12 will be directed to change its' tilt angle to 
follow the target object and will also have to 
increase its field of view (zoom out) so that the 
target object is substantially the same size in the 
image produced by camera 42 as it is in the image 
produced by the master camera 12. All of the other 
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cameras 14-40 , 44-70 will be directed to change their 
tilt angle, pan angle and/or zoom to image the target 
object 110 from their respective spatial perspectives 
and to ensure that the target object is substantially 
the same size in their respective images as in the 
image produced by the master camera 12. 

[47] If the target object 110 subsequently moves along line 
112 toward point 114, the distance D2 between the 
object of interest and camera 12 does not change, and 
assuming the that the framing size remains constant, 
the size of the object in the image produced by camera 
12 will remain the same. However, since the distance 
between the target object 114 and all of the other 
cameras 14-70 has changed, all of the other cameras 
will have to adjust their zoom to keep the size of the 
target object in their images substantially the same 
as in the image produced by the master camera 12. 

[48] An example of a general method for determining the 
geometric relationships between the cameras and the 
field, and for determining the position of a target 
object based on a camera's positioning and a height 
intercept will now be described. These relationships 
can be used by a computer program in the master 
computer (or elsewhere) to calculate positioning, 
zoom, and other camera attributes of both the master 
camera and the various slave cameras. These 
relationships form the basis of the master broadcaster 
computer's transform functions. Although a stepwise 
process is detailed below, these relationships are 
shown for exemplary purposes only, and the actual 
transform function utilized by the master broadcaster 
computer should not be limited to any particular 
methodology. 
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[49] FIG. 5 is a schematic representation of the 
relationships between a camera 12 and a target 116 on 
'the playing field. First the target f s location with 
respect to a master camera can be determined. The 
master camera tilt angle (CT) can be used to calculate 
the line of sight distance, or focal distance (FD) , 
and ground distance (GD) from the master camera, using 
the following formulas where Z is the height of the 
master camera above the X, Y plane (the field as 
determined by the calibration program) . 

[50] Z/TAN(90-CT)=GD 

[51] Z 2 + GD 2 m FD 2 

[52] Once the focal distance FD and ground distance GD are 
known, the camera pan information (CP) can be used to 
calculate the X and Y coordinates of the target. This 
is determined by finding the difference in the X 
position between the master camera location and the 
target location (DX) and the difference in the Y- 
position between the master camera location and the 
target location (DY) using the following equations. 

[53] GD*SIN(CP)=DX 

[54] GD*COS(CP)=DY 

[55] Then the camera 1 s position in the world (CX,CY) can be 
added to (DX,DY) to obtain the real world target 
coordinates of (TX,TY) . The opposite transform 
function (i.e., determining the camera tilt CT and 
camera pan CP angles from the position of a target on 
the field and the location of the camera 12) could be 
calculated by performing the reverse of the above 
equations. This may be useful in calculating the 
telemetry settings for the plurality of slave cameras 
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once the location of the target object is known (see, 
below) . 

Additionally, the target object 116 may be at some 
height above the playing field (as determined by the 
calibration program) . For example, to focus on a 
player 1 s waist, the camera 12 will be pointing at an 
intersection point with the playing field that is 
beyond the player. By using height offsets, the above 
calculations could still be used to perform the 
transform functions. 

Zoom information for the master camera 12 can be 
arrived at in two ways. Either by the master 
broadcaster computer telling the camera zoom what 
field of view is desired or the computer getting the 
information from the camera zoom. In this example we 
will have the master broadcaster computer control zoom 
based on field of view (FV) . The focal angle (FA) of 
the lens can be found using the formula: 

90-ATAN2 (FV/2 , FD) =FA 

The above steps provide all of the information needed 
from the master camera 12 . This information can now 
be used to control, the "slave" cameras, by reversing 
the calculations. First, the robotic slave camera 
positioning system must be told where to point. The 
position of the slave camera in the world coordinates 
is (CX,CY,Z) . 

The difference between the slave camera position and 
the target position (DX, DY) in the X,Y plane can be 
determined using the following formulas: 

TX-CX=DX 

TY-CY=DY 
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[63]* The difference in positions in the Z direction is 
found by subtracting the Z coordinate of the target 
(the vertical intercept) from the Z coordinate of the 
camera. Then the ground distance (GD) between the 
slave camera and the target can be found using the 
formula : 



[64] (DX 2 + DY 2 )* =GD 

[65] The Camera Tilt (CT) angle can be found using the 
formula : 

[66] 90-ATAN2 (GD, Z) =CT 

[67] The Camera Pan (CP) angle can be found using the 
formula : 

[68] 90-ATAN2 (DX,DY) -CP 

[69] The Camera Focal Distance (FD) can be found using the 
formula: 

[70] (GD 2 +Z 2 )* =FD 

[71] The Focal Angle (FA) can be found using the formula: 
[72] 90-ATAN2 (FV/2,GD)=FA 

[73] The CT, CP, FD, . and FA are then sent to the slave 



camera robotic positioning system, which will cause 
the slave camera to track the target and match the 
framing of the master camera. All of these 
calculations are preferably made using computer 
software, for example running in the master 
broadcaster computer. The above geometry-based 

discussion is used as the basis of the program, but 
any number of specific programs could be written to 
perform these functions. There are various ways in 
which this program could be written by one skilled in 
the art. 
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[74] The multiple camera video system of the present 
invention may be particularly suitable for providing 
improved video replay images during a sports 
broadcast. The video images of an event of interest 
from the plurality of, cameras will be stored in a 
suitable storage medium, such as a digital data file, 
digital disc recorders, or tape. To produce a replay 
of an event, one of the video images will be played 
until a particular video frame of the event of 
interest is depicted in the video. At that time, the 
video will be frozen to display the particular video 
frame. The display may then switch among frames in 
the video images that were produced by the other 
cameras . The frames of the video images that were 
produced by the other cameras may be frames that 
correspond in time to the originally selected frame, 
or they may be other frames, such as those recorded at 
successive or previous instants in time. Since the 
video images have been recorded from different spatial 
locations, that in the preferred embodiment encompass 
views surrounding the scene, this will effectively 
rotate or spin the object being displayed. 

[75] The cut between video sources can be done in a number 
of well-known ways, for example using a routing 
switcher such as the Grass Valley SMS 7000 or the Pesa 
Jaguar. Once a desired degree of rotation has been 
achieved, the video images from the camera positioned 
at the location viewing the desired angle of view can 
be played to continue full motion video or may be 
further edited. As an alternative to rotation of a 
frozen image, for some applications, such as where a 
single target object, or a small number of objects are 
located near the sweet spot, moving video can be 
displayed by switching among the video images from the 
various cameras . 
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[76] FIGS. 6a, 6b, 6c, 6d and 6e are simulated frames of 
video images that illustrate the operation of the 
present invention in a video replay mode. Assume that 
video images produced by camera 26 in FIG. 2 are 
displayed on a monitor. The viewer wishes to see if 
player 118 has stepped on sideline 120. As the player 
approaches the sideline, a particular frame of the 
video produced by camera 26 is selected and shown in 
FIG; 6a. In the frame of FIG. 6a, the viewer cannot 
see* if the player's left foot has hit sideline 120. 
To achieve a better view, the corresponding frame 
produced by camera 34 is shown in FIG. 6b. In this 
frame it is apparent that the player's foot has 
touched the line 120. FIGS. 6c, 6d and 6e shows the 
corresponding frame from the video recorded by cameras 
42, 48 and 56 respectively. It should be apparent 
that by switching among corresponding frames of images 
recorded by the various cameras, the image of the 
player is effectively rotated or "spun" on the 
display. Once a particular frame has been selected 
from the available frames, the full motion video can 
be resumed by running the video recorded from the 
camera that recorded the selected frame. 

[77] FIGS. 7a, 7b, 7c, 7d and 7e are simulated frames of 
video images that further illustrate the operation of 
the present invention. In these frames, the viewer 
desires to determine if the player 122 has crossed 
line 124 at the time that he crossed the sideline 126. 
FIG. 7a is a frame from the image recorded by camera 
26. From the spatial perspective of camera 26, the 
viewer cannot tell if the player has crossed line 124. 
FIG.. 7b is a frame recorded from camera 32. Here 
again, the viewer cannot tell if the player has 
crossed line 124. By switching to the frame recorded 
by camera 36, as shown in FIG. 7c, the viewer can 
clearly see that the play is to the left of line 124. 
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FIGS. 7d and 7e show corresponding frames recorded 
from cameras 48 and 58, respectively. FIGS. 7a, 7b, 
7c, 7d and 7e again illustrate how the invention uses 
a frozen image to effectively rotate the image on the 
display to provide information that might not be 
obtained from a fixed camera position. 

[78] With additional signal processing, synthesized 
("virtual") video images can be created by 
interpolating information in images taken by different 
cameras to produce views from spatial perspectives 
where cameras do not actually exist. Such synthesis 
can result in smoother transitions between images when 
the object of interest is rotated in the display. For 
example, interpolation software available from Realviz 
Corporation could be used to perform the necessary 
image interpolation. 

[79] This invention can be used as part of a live broadcast 
or as a replay device, giving a 360° view of an object 
or objects. To use the invention in connection with a 
sporting event, cameras can be installed either at 
field level or at some point above the field level 
such as the mezzanine level of a stadium. . The video 
images provided by the camera system of this invention 
are similar- to a virtual camera that can revolve 
around an object, as the object remains frozen or is 
in motion. The image produced by the master camera 
can be framed wide or tight, and the images produced 
by the additional cameras will automatically track the 
image produced by the master camera. 

[80] This invention further encompasses the detection, 
recording and playback of point specific audio. By 
placing microphones at some or all of the camera 
locations such that the microphones receive sound from 
the direction of the field of view of the cameras, 
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audio signals can be produced and fed to the computer 
for processing. Since the computer will have 
information concerning the position of the 
microphones, the audio signals produced by these 
microphones can be processed to produce an audio 
output signal representative of sound being produced 
at, a particular location in the area occupied by the 
event being recorded. 

[81] For f ' T example, since the distance from the camera to the 
object of interest can be calculated as shown above, 
if a microphone is mounted at the camera location, the 
time required for sound produced in the vicinity of 
the target object to reach the microphone location can 
be calculated. Since the microphones are located at 
various distances with respect to the object of 
interest, the sound produced in the vicinity of the 
target object will reach the microphones at different 
times. By adding a time delay to the signals produced 
by the microphones to account for the differences in 
distance from the microphones to the vicinity of the 
target object, and subsequently combining the signals 
(for example by adding the signals using a digital 
mixer) , the sound produced in the vicinity of the 
target object can be recovered. This will produce an 
audio signal having a higher signal to noise ration 
than an audio signal produced by any of the individual 
microphones . 

[82] The 'above description provided a general overview of a 
multiple camera video system according to the present 
invention and methods associated therewith. To aid in 
the understanding of the invention, and to point out 
additional features and components, the following 
description provides a more detailed explanation of at 
least one presently preferred embodiment. This 
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discussion is intended to work in conjunction with the 
above description. 

[83] FIG* 8 details a system diagram for one exemplary 
embodiment of the camera system 10 with connection to 
the camera control system 5 of the present invention. 
FIG. 11 shows how the camera system 10 is connected to 
the camera control system 5, and JFIG. 17 shows how 
this system is connected to the image storage and 
playback system 3. FIG. 8 includes system elements 
that are mounted around the target event (e.g., the 
cameras and communications devices) as well as 
exemplary communications connections to elements that 
may exist in a production truck or other site (e.g., 
master pan head and master broadcast computer) . These 
two sides of the system are denoted "stadium side" and 
"truck side" in FIGS. 1 and 8, however, most 
components may exist at almost any location given the 
proper communications connections. The camera system 
10 shown in FIG. 8 is preferably replicated at 30 
locations around the stadium. 

[84] The cameras 150, mounted around the stadium or arena, 
may be conventional video cameras, high definition 
video cameras, or any other imaging device. The 
cameras 150 are preferably mounted on a robotic pan 
head 152 or other mounting devices that remotely 
control the movement of the cameras. FIG. 8 shows a 
pan and tilt head 152 operatively coupled to the 
camera 150. A pan and tilt head 152 typically 
includes at least two axes of motion (one pan, one 
tilt) that allow control of the camera 150 over these 
multiple axes of motion. A third axis of movement, 
rotation, may also be included. 

[85] FIG. 9 shows one exemplary embodiment of a robotic pan 
head 152 for use with the present invention. As seen 
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in FIG. 9, the panning axis (horizontal plane) is 
shown directly vertical from the center of the camera 
150 (the lens focal point) . This is a preferred 
orientation because the camera lens will always be the 
same distance from this axis throughout its entire 
range of motion and no offset compensation is 
necessary as the camera 150 pans. The FIG. 9 pan head 
152 shows a tilt axis (vertical plane) that is not in 
line with the center of the camera lens. Therefore, 
as ' the camera 150 tilts through its entire range of 
motion, the lens will be at varying lengths from the 
target object, which may slightly affect the ability 
of the captured image to align with the images of the 
other cameras. It is preferred to either compensate 
for this tilt length offset in a computer program that 
processes the image or to utilize a pan head 152 where 
both the tilt and pan axes intersect the center of the 
focal point of the camera 150. 

[86] Each camera 150 has a camera control panel 162 
associated therewith which provides power to each 
camera and two-way communications between the camera 
and the control 5 and storage 3 systems (collectively 
hereafter the "truck side" of the system) . In the 
camera control panel 162 mounted near each camera 150, 
there is also preferably a pan head computer 154 that 
controls the pan and tilt of the pan head 152. This 
computer 154 preferably takes commands for remotely 
controlling the pan head 152 and translates them into 
instructions upon which the pan head can operate. The 
computer ' 154 may also translate current pan head 
positioning into a form that the truck side of the 
system 1 can analyze. 

[87] The video or other imaging signal that is captured by 
the camera 150 is converted by a video converter such 
as the fiber optic video converter 156 shown in FIG. 
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8. This video converter 156 accepts the video feed in 
a format dependent on the camera 150 and converts it 
to a signal that may be carried by a fiber optic 
cable. Such converters 156 are well known in the art. 

[88] The video converter 156 may also receive information 
from the truck side of the system 1 through a fiber 
optic cable and convert this information into a format 
that the camera 150 can understand. For example, 
zoom, focus, and paint information may be provided 
from the truck to each camera 150 to adjust for 
various objects to be targeted and environmental 
changes (e.g., cloud cover). Additionally, a genlock 
signal (described below) may be provided to each 
camera 150 so that the resulting stored video will be 
synchronized across all of the camera video feeds 
during playback. In this way, the fiber optic video 
converter 156 translates the information transmitted 
over a distance from the truck to each camera 150 into 
a format the local camera can understand (and vice 
versa) . 

[89] The camera 150, robotic pan head 152, computer 154, 
fiber optic video converter 156 and any other 
electronic components found locally near the camera 
typically need electric power to operate. FIG- 8 
shows a power connector 166 through which power 160 
may be supplied to the camera control panel 162. 

[90] FIG. 10 details a schematic view of one exemplary 
embodiment of the wiring inside the camera controller 
panel 162. FIG. 10 shows a 220 VAC power source 190 
being fed into a 12V linear power supply 192, a 24 V 
power supply 194 and a series of three 48 V power 
supplies 196 connected in series. FIG. 10 also shows 
the power connector for the computer CPU 198, the 
fiber optic video converter (shown as a Fiber Option 
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Media Converter) 200 and a fiber optic Ethernet media 
converter (shown as a Black Box media converter) 202. 
There is also an 8 pin connector 204 for the robotic 
camera mount and a camera power cable 206 (shown as a 
Sony Hirose Cable Connector for use with a Sony video 
camera) . 

[91] Looking again at FIG. 8, there is shown a fiber optic 
Ethernet media converter 158. This media converter 
158 takes the wired 182 Ethernet -based signal from the 
pan head computer 154 in the camera control panel 162 
(which is connected 180 to the robotic pan head 152) 
and converts it into a fiber optic-based Ethernet 
signal 186 for transmission to the truck side of the 
system 1. This converter 158 can also convert fiber 
optic Ethernet signals received from the truck side of 
the system 1 and to regular wired Ethernet signals 
that may be used by the computer 154 in the camera 
control panel 162. The pan head computer 154 is 
communicatively connected to the pan head 152 for the 
transmission of pan head settings to the pan head and 
for the reception of actual pan and tilt values from 
the pan head. 

[92] For ease of installation and re-installation, the 
camera controller panel 162 preferably includes a 
fiber optic connector 164 and power connector 166 for 
which to provide a power cable (in this example a 220 
VAC power cable) and fiber optic cable 168 to the 
truck or editing end of the system 1. In this 
example, three or four strands of fiber optic cable 
168 may be run together (one 184 for the camera 150 , 
two 186 for the robotic mount 152, and one spare fiber 
176). FIG. 8 shows up to a 1500 ft. run of fiber 
optic cable 168 between each camera control box 162 
and the truck side of the system. This maximum length 
is exemplary and is determined by the specific 
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components and communications schemes used. The fiber 
168 used for the bi-directional video feed is 
preferably mult i -mode fiber. This fiber is used to 
send the genlock signal and other commands up to the 
cameras 150 and to receive the video feed back from 
the camera. 

[93] It should be noted here that the fiber optic system 
shown in FIG. 8 could be replaced by a variety of 
other cabling systems. Fiber optics are shown as a 
preferred embodiment because of the distance in which 
the data can travel, but a fiber optic system is also 
more expensive, more fragile and more difficult to 
install/uninstall than other types of cable. 

[94] Alternative communications mediums could include a 
triaxial or wireless RF scheme. Many sports stadiums 
already have triaxial cable run throughout much of the 
stadium to accommodate past television technologies. 
Installation of the triax may therefore be 
streamlined. A wireless RF scheme may also be used 
but could be limited by bandwidth considerations 
created by FCC guidelines. The usable bandwidth of 
triax or glass fiber is much greater than that of the 
available RF spectrum. 

[95] Triaxial cable generally includes an inner shield, an 
outer shield, and a semiconductor. In one embodiment 
of the present invention, the outer shield can be used 
as a ground, the inner shield can be used to supply 
power supply low, and the semiconductor can be at DC 
potential. To send signals over this medium, the 
signals are AC coupled to the semiconductor to be sent 
up to the cameras. A triax adapter can be used with 
an RF modulator to send video, audio, and telemetry. 
For example, the video feed could be modulated on a 50 
MHz carrier, the audio could be modulated at 25 MHz 
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and the telemetry could be modulated at 30 MHz. . As is 
common with triaxial cable, this scheme replicates a 
wired RF communications scheme. 

[96] In much the same way, a wireless RF communications 
scheme could be used to transmit signals to and from 
the cameras. A similar transmission system to the 
triaxial system may be created, however FCC and any 
local governmental regulations must be followed for 
:: operation. Because the usable wireless spectrum is 
generally smaller than the wired (triaxial or fiber 
optic) counterpart, the wireless RF system may have to 
be based an a poorer quality signal (lower resolution) 
in order to accommodate all of the cameras on the 
reduced bandwidth. A modulation scheme in which one 
camera after another transmits video information, or 
in which only the selected master camera transmits 
information may also be utilized. However, to store 
information from each camera at all times (to 
completely archive the event) , may necessitate a lower 
quality image being captured and sent to the truck 
side of the system. 

[97] The above camera system 10 (cameras 150, robotic pan 
heads 152, camera controller panels 162, and the 
communications scheme) is preferably repeated 
throughout a plurality of camera stations around the 
stadium as described above. The example in FIGS. 2 
and 8 show 30 such camera configurations (denoted by 
»x3 0" in FIG. 8) . It should be noted that any number 
of cameras may be used. As more cameras are used, the 
amount of "skip" between images when rotating from one 
camera to the next decreases. Therefore, a high 
number of cameras is preferred, but any number of 
cameras is contemplated within the scope of the 
present invention. 
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[98] At the "truck side" (FIG. 1) of the present invention, 
there is preferably all of the components necessary to 
store and edit the video information captured by each 
of the plurality of cameras as well as the ability to 
create and communicate appropriate commands to 
instruct each of the cameras (and pan heads) as to 
positioning and camera attributes such as zoom, focus, 
various paint parameters and any other functionality 
that the user wishes to communicate to each of the 
cameras. These latter camera control system 5 
components are now described with reference to FIG. 
11. 

[99] Initially at the truck side of the system 1, there is 
a communications device capable of converting the 
signals transmitted to and from the camera into a 
usable format. In FIGS. 8 and 11, because a fiber 
optic communications system is depicted, there is a 
plurality of fiber optic Ethernet media converters 174 
(connected to the fiber optic cable 168 through a 
cable connector 170) to convert the fiber-based signal 
into a conventional wired signal. There may also be a 
fiber optic video converter 172 for the video signal 
that is not shown in FIG. 11 (described below with 
respect to the storage and playback system 3) . The 
plurality of wired Ethernet cables 111 are then routed 
through one or more hubs 210 (such as the Bay Networks 
16 Port Hub shown in FIG. 11). Depending upon the 
number of cameras 150 used (and therefore the number 
of fiber optic cables 168 routed to the truck) a 
greater or lesser number of hubs 210 are needed. FIG. 
11 shows two such hubs 210. In general, the truck- 
side communications devices (fiber optic video 172 and 
Ethernet media 174 converters) match similar elements 
(156, 158) found on the camera side of the system. 
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[100] These various hubs 210 are preferably connected 
together (212) so that any one of the camera video 
signals may be sent as an output from these hubs 210 
to one of the various truck side computers or 
components. In the same way, the Ethernet connection 
may also be used to transmit commands, coordinates or 
other information to the cameras 150 and robotic 
controller panels 162 at the camera side of the 
system . 

[101] The .master broadcaster computer 215 is one of the main 
computer systems at the truck side of the system 1. 
The master broadcaster 215 is connected to each camera 
150 through an Ethernet connection 217 to the daisy- 
chained hubs 210 and is connected to a master pan head 
220 or camera mount that may be used to control the 
positioning of the various cameras 150. 

[102] The master pan head 220 is preferably a traditional 
camera mount that can be physically panned and tilted 
to position an attached camera in a conventional 
(studio-based) video camera system. In a preferred 
embodiment of the present invention, the master pan 
head 220 has a video monitor 222 (rather than a video 
camera) mounted on the pan head 220. A camera 
operator at the master pan head 220 can move the pan 
head 220 in various directions (pan and tilt) to 
remotely control a master camera which can be selected 
from any of the cameras 150 mounted around the 
stadium, arena or other location. The use of the 
master pan head 220 provides tactile feedback to the 
camera operator . 

[103] For example, the master pan head 220 may have a 
keyboard 224 and mouse 22 6 attached thereto which 
provide additional functionality to the master pan 
head 220. The camera operator may select a "master 
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camera" from the 30 cameras 150 mounted on the 
stadium. This camera will then be remotely controlled 
by the camera operator and all of the other 29 "slave" 
cameras will be remotely controlled to follow that 
master camera (controlled by the master pan head) . 
Typically, the camera operator will select a master 
camera using the keyboard 224 (for example by typing 
in the camera number or a pressing a hot key 
associated with the camera) . The monitor 224 mounted 
on top of the master pan head 220 will then display 
the video feed from the selected master' camera. 
Assuming that camera 1 is selected as the current 
master camera, the camera 1 video feed 178 will be 
converted by the fiber optic video converter 156 in 
the camera control panel 162, will be sent across the 
fiber optic cable 168 to the truck side fiber optic 
video converters 172 , will be converted to a 
traditional wired signal, will pass through the cam-A 
computer 228 to the master pan head monitor 222. 

The camera operator will therefore see an image on the 
monitor 222 mounted on the master pan head 220 (which 
is actually the video output of the cam-A computer 
228) which is the image that camera 1 is capturing. 
As the camera operator moves the master pan head 220 
(pan and tilt) , the position data of the master pan 
head 220 is sent serially 230 to the master 
broadcaster computer 215 which then communicates this 
position data back up to the camera 1 robotic pan head 
152 (via fiber optic cable 168 through the Ethernet 
converters 174, 158). The robotic pan head 152 will 
then move camera 1 150 in accordance with the movement 
of the master pan head 22 0 by the camera operator. 
The image captured by camera 1 will then be 
transmitted through the cam-A computer 228 to the 
monitor 222 on the master pan head 220, and the camera 
operator will see the movement as if he or she was 
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standing right behind camera 1. In this way, the 
camera operator at the master pan head 220 can 
"directly" move a master camera by remote means. 

[105] As the camera operator moves the master camera with 
the master pan head 220, the master broadcaster 
computer 215 calculates pan and tilt coordinates to 
move each of the other 29 "slave" cameras in unison 
with the master camera (i.e., camera 1). Therefore, 
the master camera is directly remotely controlled by 
the camera operator and the slave cameras are 
indirectly controlled by the camera operator. The 
calculations in the master broadcaster 215 use a 
geometric transform between each of the slave cameras 
and a predefined mesh or calibration table (described 
below) . Although this transform could be coded in a 
number of different ways, the calculation basically 
translates the master pan head settings and vertical 
intercept entered by the camera operator into a point 
in space 3 -dimensional that is defined as the target 
object. The settings required for each of the slave 
cameras is then calculated using the camera-specific 
transform to convert this actual point in space to the 
appropriate camera settings {e.g., pan, tilt) as per 
the calibration table and transform. 

[106] In addition to the video signal from the cam-A 
computer 228 being sent to the monitor 222 above the 
master pan head 220 and the pan and tilt position data 
being sent from the master pan head 220 to the master 
broadcaster computer 215 (to be sent to the various 
robotic camera pan heads 152 after transformation) , 
the master pan head 220 may also include a zoom 
controller that allows the camera operator to also 
control the field of view or zoom of the master 
camera. For example, a conventional pan head zoom 
control may be a rotatable stick that can be turned 
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clockwise or counterclockwise to zoom in and out. As 
the camera operator turns the stick to zoom in and 
out, the master pan head 220 sends this zoom 
information 232 (typically as a data point 
representing a value between "no zoom" and "full 
zoom") to the master broadcaster computer 215. The 
master broadcaster computer 215 will then send this 
zoom information to' the actual master camera (camera 
1) so that this camera will appropriately zoom 
according to the camera operator 1 s control . The 
master broadcaster will also calculate the appropriate 
zoom setting for all of the slave cameras mounted 
around the stadium. This zoom correction is important 
for a "spinning" video output from one camera to the 
next (described in more detail below) . Without this 
updated zoom, the image would be enlarged and reduced 
randomly as the output is "spun" through the various 
cameras. This zoom setting calculation may be based 
on a zoom calibration table described below. 

[107] The master pan head 220 may also include a mouse 22 6 
with a wheel used to adjust the intersection height 
above ground of the target image captured by the 
master camera. As described above with respect to 
general camera geometry, the image that a camera 
captures is along a straight ray (FD in Fig. 5) from 
the camera lens to the point on the ground at which 
the camera focal line FD intersects. However, the 
camera operator may be attempting to show a football 
player carrying a football. Therefore, the actual 
target image is approximately 4 feet off of the ground 
(the ball or the player's waist), but the focal ray FD 
of the camera will intersect the ground at a point on 
the ground past the ball carrier. In order to 
appropriately adjust the focus of the master camera 
and to determine the actual target object position in 
3D space at which the slave cameras will point, the 
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camera operator may spin the wheel on the mouse 226 to 
adjust the vertical height that, is the intersection of 
this master camera focal ray FD with a vertical line 
from the ground. In the ball carrier example, a 4 
foot intersection height may be appropriate . 
Depending on the angle of the camera focal ray with 
the ground, the intersection point can be calculated 
to find a true distance from the camera lens to the 
target image (the football) . The focus on this camera 
can then be adjusted to this actual distance to the 
target object, rather than the distance to the 
intersection point with the ground. This focus 
adjustment preferably occurs based on a focus 
calibration table described below. 

[108] This height information may also be sent to the master 
broadcaster 215 so that the master broadcaster can 
determine the "actual" position in three dimensional 
space of the target object, so that the appropriate 
telemetry settings may be calculated for each of the 
slave cameras. This includes the appropriate focus 
and zoom information which must be calculated and sent 
to each of the slave cameras so that the target image 
is in focus from each of the cameras . 

[109] The cam-A computer 228 basically shows the video image . 
output from the currently selected master camera. 
This video feed is sent to the monitor 222 at the 
master pan head 220 to help guide the camera operator 
and may " further be sent to the broadcast truck to be 
sent to air or digital video storage. The cam-A 
computer 228 is preferably connected to the output of 
the Ethernet hubs 210 so that it may communicate with 
any of the cameras 150. The cam-A computer 228 
preferably also includes a keyboard 224, mouse 226, or 
other input device (which may be used locally at the 
master pan head 220) . There may also be a local 
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monitor connected to the cam-A computer 228, and a 
second monitor 222 may be connected to the cam-A 
computer 228 and reside at the master pan head 220 
(see above) . 

[110] The cam-A computer 228 may also provide a heads-up 
display (HUD) for the camera operator at the master 
pan head 220. For example, when the cam-A computer 
22 8 sends the currently selected video feed to the 
monitor 222 on the master pan head 220, the cam-A 
computer 228 may overlay crosshairs in the center of 
the video image to help guide the camera positioning. 
The HUD may also include the currently selected 
intersection height above ground for the target 
object . 

[Ill] The HUD may also include a list of positional function 
keys that are available to be selected by the camera 
operator at the master pan head 220. These positional 
function keys may be preset locations on the field of 
play, for example the pitcher's mound and each base of 
a baseball diamond. Each preset location may be set 
by aiming a camera at the selected location and using 
ESC-F1 or some other key stroke to save the location 
as positional hot key Fl. At any point during the 
broadcast, the camera operator may then hit the Fl key 
on his keyboard to position the master camera (and 
hence all of the cameras) to the selected predefined 
position. 

[112] The cam-A computer 228 may be used to select the 
output of the plurality of video feeds from the 
various cameras to determine which signal will be 
output to the broadcast truck or for the air. This 
broadcast signal is represented as "video in" 234 in 
FIG. 11. 
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[113] The master broadcaster 215 is the central computing 
facility in the multiple camera video system 1 of the 
present invention. The master broadcaster computer 
215 preferably includes a keyboard 236, computer mouse 
23 8, and/ or any other computer input device. The 
master broadcaster computer 215 is the central 
communications device for calculating and determining 
the positioning and other characteristics of the 
various master and slave computers and for performing 
various other functionalities. 

[114] The general function of the master broadcaster 215 is 
to position each of the cameras. As described above, 
a camera operator can select one of the plurality of 
cameras 150 to be the (current) master camera, thereby 
making the other 29 cameras slaves. As the camera 
operator moves the master pan head 220, the pan, tilt, 
and/or other positioning coordinates are sent by the 
master broadcaster 215 to the selected master camera 
(with or without conversion depending on whether the 
information sent by the master pan head 220 matches 
the format expected by the robotic pan heads 152) . At 
the same time, the master broadcaster 215 must 
calculate at what point in space (on the calibration 
texture map) the master camera is pointing, and 
calculate appropriate pan, tilt, and/or other 
positioning coordinates for each of the slave cameras. 
This calculation occurs because of a predefined data 
file that maps pan and tilt values for each camera to 
a texture map of the field, stadium, or other target 
surface. In this way, the master broadcaster 215 is 
the "brains" of the system. More on this will be 
discussed below after the calibration program is 
introduced. 

[115] FIG. 11 also includes one or more calibration 
computers 240 shown as calibration computer 1 and 
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calibration computer 2. These calibration computers 
240 may be used to calibrate the X,Y,Z axes of . the 
cameras 150 so that the master broadcaster can 
remotely control these cameras through the fiber optic 
network. 

[116] Each calibration computer 240 preferably includes a 
CPU, a monitor, and a keyboard 242, mouse 244 and/or 
other input device. The calibration computer 240 
preferably includes an Ethernet card or other 
communication device through which the computer 240 
can exchange data with the master broadcaster computer 
215. In the FIG. 11 example, the calibration 
computers 240 are connected to the master broadcaster 
215 through a calibration hub 246 (e.g. , Bay Networks 
16 port hub) . 

[117] The basic function of the calibration computers 240 is 
to calibrate the positioning of each of the remote 
cameras 150 enabling the master broadcaster 215 to 
remotely calculate and control the slave cameras in 
response to a movement in the master camera. Although 
this calibration is necessary before each event to be 
filmed (because the camera position will change) , the 
calibration may be repeatedly checked and updated 
during the event to ensure smooth frame transition 
from one camera to the next camera. Because the 
cameras 150 are typically heavy, the pan head 
positioning can become incorrectly calibrated after 
continual motion of the camera during the event. 

[118] The calibration of each camera preferably includes 
relating the pan, tilt, and/ or other pan head position 
settings to actual locations on the playing field or 
other target surface (actual X,Y positioning). This 
calibration is aided by a computer program running on 
the one or more calibration computers 240. 
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Preferably, there is at least one calibration computer 
240 for every five cameras to be calibrated. If a 
calibration engineer (running a calibration computer 
240) has to calibrate more than five cameras, the 
updating may not be quick enough for real-time 
calibration during the filming of the event. 

[119] A sample calibration computer program screen shot is 
shown in PIG. 12. FIG. 12 includes a window 250 for 
showing the view from a selected camera to be 
calibrated, a view 252 of the stadium, field or other 
target surface, and a plurality of control buttons 254 
for the calibration engineer to calibrate and set 
options for each camera. An exemplary calibration 
methodology utilizing a program with the FIG. 12 
screen will now be described in detail . 

[120] The calibration program preferably creates a texture 
map of the playing surface which is associated with 
actual pan and tilt positioning coordinates for each 
camera (through the use of a transform function) . 
Because each camera is at a different location around 
the field, each camera must be individually calibrated 
so that the master broadcaster 215 can point each 
camera in the appropriate location when the camera 
operator or other entity selects a certain location 
for viewing. 

[121] The first step in calibrating the cameras is to 
generate and store a set of data points for various 
locations around the playing surface of the target 
field. These data points should be real -world 3- 
dimensional coordinates that define the field of play. 
For example, an X,Y grid system in a horizontal plane 
could be used in conjunction with an azimuth or Z axis 
in the vertical direction. For generally flat playing 
surfaces (such as a tennis court) , the Z axis may be 
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estimated as flat, but this vertical axis may be 
important for crowned playing surfaces such as a 
football field or for uneven playing surfaces such as 
the pitchers mound of a baseball field. By capturing 
these real world coordinates in a texture map, and 
then correlating pan and tilt settings for the robotic 
pan head of each camera to these coordinates, a 
transform function can be written using basic 
geometric relationships to take any point in three 
dimensional space and calculate pan and tilt settings 
to aim a particular camera at that point. 

[122] The initial playing field mapping is carried out by 
selecting a plurality of points around the field and 
determining the actual coordinates in space. At each 
of a plurality of points, a theodolite, GPS global 
positioning device, or some other positioning device 
may b 4 e used to generate the absolute positioning 
coordinates. For example, an electronic theodolite 
may be moved from one point to the next and the 
absolute X,Y,Z coordinates may be automatically stored 
for later downloading into the calibration computer 
240. Alternatively, a global positioning device may 
be placed at the series of points and the absolute 
positioning may be stored. A sample text file for the 
absolute (raw) data points is shown in FIG. 13 for a 
texture map made from 54 calibration points. 

[123] The number and placing of these calibration points may 
vary, but the texture map will be more accurate as the 
number of calibration points increases. Additionally, 
selecting easily identifiable (i.e., repeatable) 
points (such as the various corners and intersections 
of lines on a tennis court) make the calibration of 
each camera 150 easier. If each of the cameras 150 
can not be directed at exactly the same calibration 
points, there may be some discrepancies between where 
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each of the cameras 150 is pointed when the transform 
functions are employed. Therefore, accurate 

calibration may be important. 

[124] With reference to PIG. 12, a schematic picture of the 
field of play 252 (in this case a tennis court in a 
tennis stadium) may be downloaded to the calibration 
program and utilized as a quick reference for the 
acquired calibration points. Each of the acquired 
calibration points from the theodolite, GPS device, or 
other device are then mapped onto this schematic of 
the field of play 252 and represented as some symbol 
(such as a square) . This bit map picture will not be 
used by the calibration computer, but may be useful to 
the calibration engineer for a quick reference of 
which point he is calibrating. The actual pan and 
tilt settings of each camera 150 must then be mapped 
to these absolute position coordinates for each of 
these calibration points so that a mathematical 
transform between each of the cameras and the absolute 
positional coordinates of the coordination points can 
be created for each camera. 

[125] After these absolute date points are captured, each of 
the plurality of cameras must be aimed at the location 
of each of the points, and the pan and tilt settings 
must be captured. These pan and tilt settings can 
then be used to generate the transform. 

[126] The calibration engineer initially selects a first 
camera (e.g., camera i) for calibration. The 
calibration engineer may select this camera by a drop- 
down menu 256 on the calibration program or by some 
other means. Preferably, the view from the selected 
camera will then be displayed in the camera display 
window 250 on the top of the calibration program 
display. Utilizing the keyboard 242, mouse 244 or 
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other input device, the calibration engineer 
preferably is capable of panning, tilting, zooming 
and/or focusing the selected camera to calibrate the 
camera . 

[127] The calibration engineer moves the camera to the first 
point captured by the theodolite, GPS device or other 
device. Preferably, the camera is zoomed to the 
maximum extent possible to make the point capturing as 
accurate as possible. As seen in FIG. -12, there may 
be crosshairs 258 on the camera display window in 
order to guide the calibration engineer in "aiming" 
the camera. Once the crosshairs 258 are aligned with 
the first calibration point, the actual pan and tilt 
coordinates of the robotic pan head 252 of the 
selected camera are preferably sent to a text file or 
other data file. Also, the vertical or Z coordinate 
may be read from the theodolite or absolute coordinate 
already captured. This data capturing may be 

implemented, for example, by selecting the "save" 
button 260 on the calibration software display. 
Saving this value inserts the pan, tilt and/or other 
settings into a text or data file such as that shown 
in FIG. 14 for camera 1. There is preferably a camera 
data file such as this for each of the cameras, and 
this data file is used to generate the geometric 
transform function used by the master broadcaster 
computer 215 to remotely control each of the cameras. 
The data files may also include a zoom table and focus 
table (described below) , as well as any other 
pertinent information. 

[128] The pan, tilt, height (Z axis), focus and zoom may 
also preferably be directly entered by the calibration 
engineer by using the slider bars 262 or directly 
entering the data values 264 from the keyboard into 
the software. After the data values for the first 
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calibration point are captured into the camera 1 data 
file, the calibration engineer moves the selected 
camera to the second calibration point. Again, the 
camera is preferably zoomed to its maximum extent to 
generate the most accurate values for the . pan and 
tilt . Once the point is centered in the crosshairs 
258 of the calibration display window 250, the point 
is written to the camera 1 data file as the pan and 
tilt settings for calibration point number 2. 

■ 

[129] All 54 - (or any defined number) of the calibration 
-points are preferably entered into the camera 1 data 
-"file this same way. The program is capable of 
creating the appropriate transform with less than all 
54 points captured, but the more points entered, the 
better the resulting texture map transform function 
will be. 

[130] Once a sufficient number of points have been stored 
for the selected camera, the calibration engineer 
creates the texture map by selecting the "geometric 
calibration" 2 66 or other button on the calibration 
software display. This button 266 preferably 

initiates a program which calculates a geometric 
transform based on the various geometric relationships 
between each camera and the calibration points and the 
captured pan and tilt values. These geometric 
relationships are described in more detail above, but 
basically enable the master broadcaster 215 to use the 
transform values to calculate a pan and tilt setting 
for a camera based on a point in three dimensional 
space, and vice versa, if necessary. 

[131] There may also be a "mesh" functionality 268 that 
provides either a localized or field-wide texture mesh 
of the playing surface. The meshing function 268 
draws a computer mesh between selected points to 
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provide greater resolution at all . points in the field 
for the transform functions. These local or field- 
wide meshes take into account height variations at 
different parts of the field or stadium (rather than 
assuming a flat surface) . These meshes may be useful 
on a field-wide basis for uneven playing surfaces such 
as the crowned surface- of a football field. The mesh 
can also be used for localized areas of an otherwise 
flat field such as the pitcher's mound of a baseball 
stadium. 

[132] To initiate a mesh calculation, the calibration 
engineer preferably selects a plurality of calibration 
points using the computer mouse 244 or keyboard 242 
and then selects to "apply mesh" from a button 268 or 
other selection device on the display of the 
calibration program. This selection initiates the 
mesh program and the computer mesh is created by 
software which the master broadcaster 215 uses to 
calculate camera pan and tilt values (from the 
transform) when the master pan head 220 is moved 
during filming. Using a field-wide mesh is preferred 
because of the higher resolution calibration settings, 
but may be used only locally if the computing power is 
not sufficient to perform such detailed calculations 
continuously during taping. 

[133] There may also be an "initialize camera" button 270 to 
reboot the camera in case of a failure or other 
problem. When the system is first turned on, each of 
the cameras goes through a boot sequence to prepare it 
for use. If one camera functions improperly during 
taping, it may be remotely rebooted from the 
calibration station using this feature. This may 
reduce the necessity of having an engineer go out to 
the camera to initiate a reboot. 
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[134] Once this first selected camera has been calibrated 
geometrically, by the mesh method or by a combination 
of both methods, the camera can be released (e.g., by 
selecting the "release camera" button 272) which puts 
the camera back "on-line" as a slave camera in the 
system. Assuming the master pan head 220 is pointing 
at a different spot in the stadium than the camera 
that is being calibrated is pointing, this released 
camera will reposition itself to that spot and 
continue normal operation. 

[135] A second camera is then selected by the drop-down menu 
256, keyboard 242 or some other device, and this 
second camera is taken off-line and calibrated in the 
same way as the first camera. Preferably, the same 54 
calibration points are selected and entered into a 
data file, and a geometric and/or mesh calibration map 
is created. The only difference will be that a 
different data file (now for camera 2) will be 
created. All other aspects are preferably the same as 
those described above. 

[136] Each of the cameras needs to be initially calibrated 
upon installation at a specific location. 
Additionally, throughout the program, the camera 
calibration must be updated as the robotic pan heads 
152 lose their calibration ("drift") and other 
environmental aspects affect camera positioning. 
Because cameras can be taken "off-line " one at a time, 
re-calibrated, and then released to the system, the 
re-calibration may continue throughout filming. It 
has been found experimentally that limiting the number 
of cameras per calibration computer 240 to no more 
than five is preferred. Additional calibration 
computers can be used for additional cameras. 
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[137] Either during the initial calibration of each camera 
or at any other time, the calibration engineer may 
also setup and calibrate a zoom table and focus table 
for each camera. As seen in FIG. 14, the zoom and 
focus tables preferably reside in the same data file 
as the pan and tilt values stored in the data file for 
calibration. In additional embodiments, the zoom and 
focus tables may reside in separate data files . 

[138] A conventional video camera has a zoom control that 
can be taken from some minimum value to some maximum 
value. For. robotic control or sensing of the zoom 
position in the camera, .the various amounts of zoom 
are associated with a data number, such as a number 
between 0 and 255. Therefore, the minimum zoom may be 
at zoom=0 and the maximum zoom may be at zoom=255 . 
Because the size of the target object should always be 
consistent in all of the cameras (which will all be at 
different actual lengths from the target object) , the 
system preferably keeps track of the appropriate zoom 
by utilizing the zoom table. 

[139] When the camera operator utilizes the master pan head 
220 to point the selected master camera at a target 
object, the camera operator can also zoom in of out on 
the object utilizing the zoom controller described 
above. If the captured image is viewed on a 
television monitor, the target object will be of a 
certain size, depending upon the amount of zoom 
selected. Using the multi-camera system 1 of the 
present invention, the ability to rotate from one 
camera to the next is desired. However, in addition 
to having all of the plurality of cameras 150 pointing 
at the same object, each of the cameras 150 must also 
have an appropriate amount of zoom so that the target 
object appears at a near equivalent size. Without 
this constraint, the target object would appear to 

44 



WO 03/041411 



PCT7US01/46357 



reduce and enlarge in size as the image is rotated 
. throughout the various cameras 150. However, because 
each of the cameras 150 is at a different actual 
distance from the target image (because the cameras 
150 are mounted at different places around the outside 
of the stadium) , the zoom table is used to adjust for 
the appropriate zoom length for each of the cameras. 

[140] The zoom table is created by associating a specific 
zoom value (e.g., 0 to 255) with the field of view 
(camera image width) shown in the camera 150 for that 
particular zoom value. The- zoom value can be read 
from the robotic pan heads 152 (or sent to the pan 
heads) as an 8 bit word over the fiber optic Ethernet. 
To "calibrate" a zoom point, the camera is pointed as 
level as possible and a specific zoom value is 
selected. The 'camera is then panned left and right 
and the encoders are read off of the pan head 152 to 
determine the width of the field of view. The tilt 
can then be used to determine the height of the field 
of view in the same way, if desired. 

[141] PIG. 14 shows a field of view value associated with 
each of the 255 camera zoom values,, but not every 
point needs to be actually measured. Instead, a 
limited number of sample zoom value/field of view 
measurements can be taken, and a software program can 
be used to interpolate the values of the other zoom 
values by plotting the known (experimentally found) 
values and plotting them on a curve against the zoom 
settings. In this way, a table such as the zoom table 
shown in FIG, 14 can be set up for each of the 
plurality of cameras 150. 

[142] In actual use, the zoom table is used to adjust the 
field of view of each of the cameras 150 in the system 
1. If the camera operator uses the master pan head 
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220 to point at a target image with a specific zoom, 
these positional coordinate values and zoom value will 
be sent to the master broadcaster computer 215 (as 
well as to the master camera so that the robotic pan 
head can position the camera) . The master broadcaster 
215 then calculates (described above) the actual 
distance from the master camera to the target image as 
well as the field of view value at that distance and 
the selected zoom value. The master broadcaster can 
then use the zoom table associated with the other 
cameras to "reverse calculate" an appropriate zoom 
value for each of the plurality of slave cameras. As 
described above, the distance from the first slave 
camera to the target image can be calculated using the 
vertical intercept method. The zoom table for that 
slave camera can then be used to select an appropriate 
zoom value so that the field of view for the slave 
camera matches the field of view for the selected 
master camera. If these zoom values match, the target 
image will be of approximately similar size when 
images from the two cameras are viewed in succession. 
This same zoom calculation is preferably implemented 
for each of the slave cameras when their location is 
calculated by the master computer. 

[143] There may also be a focal table associated with each 
of the cameras. As shown in FIG. 14, the "range vs. 
focus table" works in a way similar to the zoom table. 
For ease of use with conventional video cameras, it 
may be preferred to create this table as an inverse of 
the range versus the focus parameter of each camera. 
To set up the focus table, an object is brought into 
focus at various distances (ranges) from the camera 
and the inverse range and focus parameters are charted 
in the focal table. As more points are gathered, the 
resolution of the focal table will increase. This 
focal data table may exist as part of the camera data 
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table (as in FIG. 14) , or it may exist as a separate 
data table. 

[144] As the camera operator utilizes the master pan head 
220 to move the selected master camera to follow a 
target object, the master broadcast computer 215 uses 
the transform function to calculate the pan and tilt 
settings for the plurality of slave computers. As 
described above, the actual distance from the target 
object to each of the plurality of slave cameras can 
be calculated with software. Once this actual value 
is calculated, the focal table can be used to look-up 
an appropriate value for the focal parameter for each 
of the video cameras. 

[145] Although not shown in PIG. 8, the multi-camera video 
system of the present invention may also include one 
or more paint stations that allow a paint station 
operator to manipulate certain other features of the 
plurality of video cameras, such as the iris level, 
shutter, and/or color settings. Each paint station is 
preferably connected to the master broadcaster 
computer 215 such that the images captured from each 
of the cameras 150 can be selectively viewed on the 
paint station computer, and the paint station computer 
can send paint, iris and other commands to each of the 
cameras 150. The paint station preferably includes a 
paint computer with monitor, a keyboard and a mouse or 
other input device. 

[146] FIG. 15 details one exemplary embodiment of a paint 
station computer program display 300 for implementing 
the functionality of the paint station. The paint 
station program preferably includes the ability to 
select one or all of the plurality of cameras, the 
ability to control the iris, shutter, exposure, or 
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color of each of . the cameras, as well as other 
functionality in the cameras. 

[147] To select a camera for adjusting, the paint station 
operator may type the camera number into a dialog box 
302 or press a hotkey on the keyboard. Once selected, 
the camera can be controlled by the paint station 
program. Unlike with the calibration program, the 
paint station program preferably does not necessitate 
the movement of the camera, so the camera does not 
need to be taken off-line to adjust it. Instead, the 
camera can be adjusted "on the fly" while still 
recording images and being controlled (for positional 
purposes) by the multi-camera video system computers. 

[148] Once the camera is selected, the paint station 
operator can manipulate the camera. For example, 
moving the slider or entering a parameter value in the 
"manual iris" section 304, the operator can remotely 
open or close the iris on a camera to let in more or 
less light. This may be useful when a part of the 
stadium loses some of its light for some reason. If 
only one camera seems to be too dark, the paint 
station operator can flip back and forth between the 
two cameras and adjust the iris on the darker camera 
until the two cameras have a near equivalent 
brightness. This ease of selecting different cameras 
may be preferred. 

[149] In much the same way, the shutter value 306 can be 
changed. For example, if a slow motion replay is 
anticipated, the shutter speed can be increased to get 
a better resolution of images at a slower playback 
speed. The color section 308 preferably allows the 
paint 'station operator to manually adjust the red, 
green, and blue color settings of each camera. This 
section 308 could be adjusted to work with any other 

48 



WO 03/041411 



PCT7US01/46357 



type of color input used by various video or non- video 
cameras. Again, the images from two different cameras 
could be rotated back and forth between each other 
until the proper color adjustments are made. 

[150] There may also be additional controls such as zoom 
310, focus 312, master pedestal 314, detail 316, gain 
318, exposure 320 and others that can be updated form 
the paint station. These various feature adjustments 
preferably work in a manner similar to those described 
above. Also, these adjustments are sent to the 
cameras 150 in real-time, through the master 
broadcaster 215 and the fiber optic network. Because 
the cameras 150 may still be "on-line," it is 
preferred to adjust those cameras that are not 
currently outputting an image signal directly to air 
(broadcast) or the image may appear strange as it is 
changed . 

[151] In addition to the "single camera" changes just 
described, there may also be the ability for the paint 
station operator to update an attribute of each of the 
plurality of cameras at the same time. For example, 
if a large cloud flies over a tennis stadium during 
the filming of a match, the amount of light picked up 
by every camera will be reduced. Rather than 
adjusting each of the cameras separately (which may 
take an extended period of time) , the paint station 
operator can utilize the "all" selection button 322- to 
adjust every camera at once. The paint station 
operator can then open the iris of whatever camera he 
or she is currently viewing by moving the slider bar 
for manual iris 304. The paint station computer can 
then calculate the percentage of change that was made 
to the iris (e.g., opened by 10%) and then make the 
same change to the iris values of all of the other 
cameras. Although an absolute amount of change could 
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be made to all of the cameras, the percentage change 
is preferred because the different cameras will have 
different iris settings when viewing the same object 
because the cameras exist at different location with 
varying amounts of ambient light. 

[152] This ."all" camera adjustment 322 may also be used for 
those featured described above such as shutter 3 06, 
color 308, exposure 320, zoom 310 and focus 312. 
There may also be a selection mechanism whereby a 
selected subset of cameras could be manipulated in 
unison. This may be useful as the sun sets below the 
edge of the stadium, whereby only those cameras on one 
side of the stadium need to be adjusted as the other 
cameras are still in the light. 

Because of the time needed to adjust these features, 
it is preferable to have no more than five cameras 
assigned to any one paint station computer. If more 
than five cameras are utilized in the system 1, there 
is preferably an additional paint station. Also, as 
the temperature in the stadium and other environmental 
conditions change throughout the game, the paint 
station-controlled parameters should be updated 
throughout the filming process. The ability to adjust 
these parameters while the system is running . aids in 
this process. 

[154] The above description details how to position and 
remotely control various features of a plurality 
cameras 150 mounted around a target image. Because 
the zoom, focus and other features are controlled, 
various image storage and replay systems may be used 
to create effects that can be broadcast over 
conventional television or other mediums. Some 
exemplary embodiments of these storage and replay 
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systems, which were introduced above, will now be 
described. 

[155] The fiber optic video converters 172 at the truck side 
of the system (shown in FIGS . 8 and 11) preferably 
include analog outputs. As described above, this 
analog output may be sent to various components of the 
robotic control system 5 (e.g. , cam-A 228, calibration 
computers 240) to provide visual feedback of the "aim" 
of each camera 150. As described below, these outputs 
are also sent to a plurality of digital disc recorders 
(DDRs) or other storage devices to be archived and 
used to "produce" multiple camera video feeds to be 
broadcast over the air. This storage scheme will now 
be described. 

[156] FIG. 16 generally shows a DDR-based image storage and 
playback system 3 to be used in the multiple camera 
video system 1 of the present invention. FIG. 16 
includes the plurality of fiber optic video converters 
172 (from FIGS. 8 and 11) connected to the video feed 
of each of the plurality of cameras 150. In this 
embodiment, the video converters 172 are connected to 
a single strand of glass fiber 188 (see, FIG. 8). The 
genlock signal 342 that was previously described as 
being sent to each of the cameras 150 is also fed into 
each of the fiber optic video converters 172 to make 
sure that each field of each frame of stored video 
from each camera are synchronized in each of the DDRs. 
This genlock signal 342 "slaves" all of the DDRs 346 
together by virtue of the time code. In the present 
example, there are 30 fiber optic converters 172 
shown . 

[157] More specifically, the genlock signal 342 is used to 
make sure all of the cameras 150 and DDRs 346 work 
together to capture and store images from each of the 
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plurality of cameras 150 at the exact same instant in 
time. The genlock signal 342 uses a time code that 
breaks down each television frame into its constituent 
fields so that each camera 150 and DDR 346 is exactly 
synchronized. A traditional frame synchronizer, on 
the other hand, only synchronizes by frame, not by 
constituent field. Therefore, using such a frame 
synchronizer only assures that each camera and DDR are 
in the same television frame, not the ■ same field 
within each frame. Allowing the cameras 150 to record 
in different fields may cause the resulting image to 
"flutter" when the image is rotated from one camera to 
the next. Each TV frame typically includes 4 fields, 
and not synchronizing by field can cause a noticeable 
flutter, especially when the slow-motion controller is 
used to spin through multiple camera video feeds. 

[158] There is also a plurality (in this case 30) of analog 
distribution amplifiers 340 connected to the output of 
each of the fiber optic video converters 172. These 
amplifiers 340 take the analog output of the video 
feed from the fiber optic video converter 172 and 
strengthen the signal while providing additional 
analog outputs for further distribution. The outputs 
of these amplifiers can then be fed to the DDRs 346 or 
other storage devices, the cam-A computer 228, the 
paint stations 360, the calibration computers 240, any 
monitoring stations 362, and/or other elements of the 
system. 

[159] One storage element for use with the present multi- 
camera video system is a digital disc recorder 346, 
such as the DoReMi DDR with 2:1, 4:1, or 8:1 
compression. The DDR 346 typically includes both 
analog and digital inputs as well as an analog to 
digital converter built into the analog input so that 
analog input data can be stored in a digital format. 
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[160] The output of each of the analog distribution 
amplifiers 340 may be brought either directly or 
indirectly into the DDRs 346. Because the video feed 
is analog at this time, it can be connected directly 
to the analog input of the DDR 346 and the internal 
analog-to-digital converter (typically an 8 bit 
converter) can be used to digitize the information so 
that it can be digitally stored in the DDR 346. 
Alternatively, each of the 30 (or any number) analog 
signals from the distribution amplifiers 340 could be 
run' through an analog- to-digital converter 348, such 
as the 10 bit converter shown in FIG. 16. The output 
of each of these analog-to-digital converters 348 can 
then be brought into the digital input of each of the 
DDRs 346. These 10-bit analog-to-digital converters 
may provide a higher resolution digital signal to the 
DDR 346, but it also adds an additional cost to the 
system when compared to using the analog input on the 
DDRs 346. Also, the higher resolution signal will use 
up more DDR storage space per unit of recording time 
than a lower quality version of the same information. 
Therefore, there is a tradeoff between image quality 
• and the amount of time for which the signal can be 
recorded. 

[161] Additionally, the DDRs 346 may include an internal 
compression algorithm that allows compression of the 
digital data. For example, the DDRs 346 shown in FIG. 
16 include a 2:1, 4:1 and 8:1 compression. Again, the 
tradeoff is between recording a longer amount of time 
and the quality of the recorded image. For a 
broadcast signal, the compression should preferably 
not exceed 2:1. 

[162] The output of each of the plurality of DDRs 346 is 
then preferably connected to the input of a digital 
router 350 that is controlled by a slow motion 
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controller 354. The digital router includes enough 
inputs so that a digital feed from each of the DDRs 
346 (e.g., 30) may be connected thereto, and at least 
one output which is preferably connected to an 
additional DDR 352 or other storage device. The slow 
motion controller 354 is preferably a modified tape 
controller that acts as a router controller for 
controlling which of the DDR feeds is passed out of 
the router 350 for storage in the additional DDR 352. 
The output of the additional DDR 352 is then available 
to be sent to air or stored for further production. 
Although it appears that the controller 354 is only 
controlling this additional DDR 352, it is actually 
controlling each of the plurality of video storage DDR 
346 outputs simultaneously. The genlock signal 342, 
described above, allows each of the DDRs 346 to be 
completely synchronized. 

[163] The slow motion controller 354 preferably includes the 
ability to move forward and backwards through the 
recorded video content of the DDRs 346 and to "rotate" 
through the different cameras 159. If the action is 
frozen in a video picture frame and the router output 
350 is rotated from one camera to the next camera and 
then to the next camera, the images output from the 
digital router 350 will appear to rotate around the 
target object (as generally described above) . These 
motion and camera selectors may be slider bars, 
joysticks, knobs or other selectors. They may exist 
in one controller, or there may be a separate camera 
selection controller (with rotating knob) connected to 
the slow motion controller 354 for moving forward and 
backwards through the video feed. 

[164] This additional DDR 352 output may be sent directly to 
a broadcast truck or other entity to be broadcast to a 
live television audience or it may be recorded to yet 
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another DDR 356, tape, or other storage device. If an 
additional storage device 356 is used, a second tape 
controller (slow motion controller) 358 can be used to 
further edit or produce the stored segment of video. 
For example, the first slow-motion controller may 
select the view of a quarterback as he drops back and 
hands off to a running back and then rotate around the 
running back as the ball carrier . fumbles the football. 
The segment may continue from a second camera's point 
of view as the teams scramble for the loose ball, and 
a third camera may be used to show the defensive back 
pick up the fumble and run in for the touchdown. 
While one network may choose to show this pre-produced 
video segment (saved on the first DDR 352) to their 
audience (for example, on the nightly news), the 
network that is currently broadcasting the game may 
choose to record the segment to an additional DDR 356 
and then further analyze the play. By using a second 
slow-motion controller 358, the point of fumble can be 
rewound and replayed multiple times, at varying 
speeds. The output of this additional DDR 356 could 
then be fed to a broadcast truck or additional 
storage. This second slow-motion controller 358 
provides additional production flexibility, 

[165] Looking at FIG, 16, the output of the analog 
distribution amplifiers 340 preferably also go to an 
analog router 344 that distributes the analog video 
feeds to a plurality of elements on the robotic side 5 
of the system 1. For example, the analog router 344 
preferably includes a plurality of outputs, as . shown 
in FIG. 16. Each of the analog outputs may be 
controlled from a different source by a rotatingly 
selectable switch, such as a BUF controller. By 
rotating the switch at the remote location, the 
different camera video, feeds may be selected. As 
shown in FIG. 16, the analog outputs of the router 344 
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may be fed to the monitor on the master pan head from 
cam-A 22 8, each of the calibration stations 240, each 
of the paint stations 360, any monitoring station 362, 
and/or any other element that utilizes selectable 
control of the video feeds. This demonstrates the 
intermixing of the components between the camera 
control system 5 and the image storage and playback 
system 3 . 

[166] In another embodiment of the present invention, the 
multiple digital disc recorders 346 could be replaced 
by a file server or some other mass storage apparatus. 
As described above, the DDRs 346 have a single output, 
and therefore can only be used by a single user at a 
time. The advantage of a file server or other mass 
storage device is that multiple entities can be 
streamed the content simultaneously, thereby allowing 
for multiple feeds of the same content. For example, 
multiple networks could receive a feed for an 
interesting play at a football game and each generate 
their own broadcast effects based on the recorded 
information. Additionally, the information on the 
file server can be easily copied to multiple output 
sources thereby aiding in the archiving and 
distribution of the feed. 

[167] For certain effects that may be difficult, to produce 
directly from a file server, for example a reverse 
motion play) , the file server contents could be 
recorded to a tape. This tape could then be used like 
the DDRs or other conventional devices to create 
reverse motion or other effects. 

[168] In an extension of this concept, the images (server 
contents) could be streamed to an end user over the 
Internet through a device such as an Internet -enabled 
video game machine, a home computer, or other Internet 
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computes the position of the target object on the 
field in order to remotely control the master and 
slave cameras 150. This positioning calculation can 
also be used to calculate the distance from each 
directional microphone 402 to the target object. The 
digital mixer 404 then can delay the input from each 
microphone 402 an amount of time that compensates for 
the propagation of sound. To be even more accurate, 
the system preferably calculates the appropriate speed 
of sound based on the current relative humidity at the 
stadium. During a sporting contest or other event, 
this relative humidity will change and therefore 
should be compensated for. Additionally, the altitude 
of the event should be taken into account. By 
delaying the "quicker" microphones, the sound from 
each of the microphones 402 will be in phase and the 
sound will be added from all of the microphones. This 
calculation may be carried out on a microphone 
computer 406 connected to the digital mixer. 

[173] The multiple camera video system 1 of the present 
invention could also be enhanced with the use of 
"virtual" cameras inserted between the plurality of 
actual cameras 150. If less than a desired number of 
camera positions are available, or if a greater 
resolution of image rotation is desired, additional 
"virtual" cameras could be implemented using a 
software program to interpolate camera images in 
between two existing cameras. For example, FIG. 18 
shows one corner of a football stadium 410 with three 
cameras 150 mounted on robotic pan heads 152 according 
to the present invention. FIG. 18 also shows the 
location of a virtual camera 912 that is mounted 
between each two "real" cameras 150. This virtual 
camera 412 does not physically exist, but the image 
from such a camera 412 could be fabricated with 
software . 
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[174] For example, as images are captured by the two real 
cameras 150 and sent to the broadcast side of the 
system (e.g., the master broadcaster 215), a computer 
may use software to compare the pixels of each frame 
from one camera with the corresponding pixels in the 
adjacent camera. To create a "virtual" camera 412 
between these two real cameras 150, the software 
preferably interpolates the color of each pixel to be 
a transition color between the colors of the two 
actual pixels. If the two cameras 150 have pixels of 
the same color, the virtual image will have that same 
color. If the two cameras 150 have different colors, 
these colors are analyzed and a transition color is 
used. This transition color may be determined based 
on the values for red, green, and blue in the color 
spectrum, or by some other means. 

The resulting virtual image does not necessarily 
represent any real-world image that could be taken. 
Instead, the virtual image provides a smoother 
transition between images when the image is rotated 
from the first real camera to the second real camera. 
Therefore, when spinning through the cameras 150, 412, 
the resolution is increased. 

Although this feature has been described with the 
production of a singe virtual image 412 between two 
adjacent real cameras 150, there could also be two or 
more virtual images 412 interpolated in this gap. The 
color transition may be scaled to utilize any number 
of virtual images, and the present invention is not 
limited to any particular number of virtual images. 

[177] It should be noted that the master camera positioning 
device has been described above as being a 
conventional manual pan head that provides tactile 
feedback to a camera operator while he or she remotely 



[175] 



[176] 
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controls a master camera. In actuality, any device 
that generates location information that can be 
translated or transformed into camera and robotic pan 
head settings (i.e., pan, tilt, etc.) may be used. 
For example, a- joy stick or any other input device may 
be manipulated to move the master camera. 

[178] In one embodiment of the present invention, there is 
not even a need to directly control the master camera 
through manual means. For example, an RF tracking 
system could be used. In this system, each of the 
players, the ball, or any other object could include a 
small RF transmitter or transceiver that outputs 
positional coordinates as the player or ball moves 
around the field. These positional coordinates may 
then be received by a receiver connected to the master 
broadcaster computer. This master broadcaster 

computer can translate the positioning coordinates 
into actual pan and tilt settings for each of the 
master and slave robotic pan heads. The positional 
coordinates of each player (from each transmitter) may 
be sent over a different carrier frequency, so that 
one specific target transmitter could be chosen. For 
example, the camera operator may select the target 
transmitter by using preset function keys tied to each 
transmitter in the same way that preset function keys 
were used above to select different areas of the field 
(pitchers mound, etc.). This RF scheme requires even 
less manual control than other some other embodiments 
described above. 

[179] With this invention, a director . of a televised 
sporting event can produce replays of the events that 
permit an in-depth analysis of the action by combining 
information contained in video images taken from 
different spatial perspectives. The invention also 
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provides the ability to change video framing during 
recording of an event. 

[180] In addition to the use of the present invention in 
broadcasts of football games as shown in the preferred 
embodiment, this invention has many other practical 
applications such as, golf swing analysis and 
telecasts of basketball, gyjnnastics, track and field, 
boxing and entertainment events. The invention could 
also used in the movie industry. 

[181] For example, rather than filming a movie scene 
numerous times from various different angles, a 
multiple camera system could be used to follow a 
target object in the movie scene using a remote master 
pan head and other system components. Such a system 
may save studio time in the production of movies and 
may further enable a smoother transition from one 
camera angle to the next. 

[182] Additionally, by using high definition video cameras 
in place of traditional film cameras to record a scene 
from a multiple of perspectives, a more complete 
record of the scene can be obtained, providing for 
more options in editing of the video for incorporation 
into a final video sequence. For example, a 16:9 
aspect ratio high definition camera with a wide angle 
lens could be used to capture the target image from a 
plurality of locations. Then, each of the wide angle 
images could be broken up into two, three or more 
images that reside next to each other in real space. 
This utility can be accomplished by segmenting the 
video frame into different sectors. A "spinning" 
output could then be produced using a reduced number 
of cameras than described above. 

[183] Nothing in the above description is meant to limit the 
present invention to any specific materials, geometry, 
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or orientation of elements. Many part /orientation 
substitutions are contemplated within the scope of the 
present invention and will be apparent to those 
skilled in the art. The embodiments described herein 
were presented by way of example only and should not 
be used to limit the scope of the invention. 

[184] Although the invention has been described in terms of 
particular embodiments in an application, one of 
ordinary skill in the art, in light of the teachings 
herein, can generate additional embodiments and 
modifications without departing from the spirit of, or 
exceeding the scope of, the claimed invention. 
Accordingly, it is understood that the drawings and 
the descriptions herein are proffered only to 
facilitate comprehension of the invention and should 
not be construed to limit the scope thereof. 
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CLAIMS 

WHAT IS CLAIMED IS: 

1. A multiple camera video system, comprising: 
a plurality of cameras; 

a master pan head for positioning a selected master 
camera from said plurality of cameras; and 

a master broadcaster computer for calculating 
telemetry for at least one slave camera from said 
plurality of cameras. 

2. The system of Claim 1, wherein said master pan 
head is remote from said plurality of cameras. 

3. The system of Claim 1, wherein said master pan 
head includes a monitor mounted thereon. 

4. The system of Claim 1, wherein said master pan 
head includes a zoom adjustment. 

5. The system of Claim 1, wherein said master pan 
head includes a height intersect adjustment. 

6. The system of Claim 5, wherein said height 
intersect is selected using the wheel on a computer 
mouse . 



7. The system of Claim 1, further comprising: 
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a plurality of robotic pan heads upon which each of 
said plurality of cameras is mounted for remotely 
controlling said plurality of cameras. 

8. The system of Claim 7, wherein said robotic pan 
heads include a pan and tilt function. 

9. The system of Claim 8, wherein the pan and tilt 
axes of the robotic pan heads intersect at a point 
within the body of said plurality of cameras. 

10. The system of Claim 1, further comprising: 

at least one paint station connected to said master 
broadcaster computer. 

11. The system of Claim 10, wherein each of said at 
least one paint stations comprises: 

a monitor; 

an input device; and 

a paint station computer running paint station 
software . 

12. The system of Claim 11 , wherein said paint 
station is capable of adjusting an attribute of at 
least one of said plurality of cameras. 

13. The system of Claim 12, wherein said attribute is 
selected from the group consisting of red paint, green 
paint, blue paint, shutter, iris, zoom, and focus. 
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14. The system of Claim 12, wherein the paint station 
can adjust said attribute on more than one of said 
plurality of cameras simultaneously. 

15. The system of Claim 12, wherein said camera 
attribute can be adjusted while the camera telemetry 
is being automatically controlled by the master 
broadcaster computer. 

16. The system of Claim 12, wherein the number of 
said at least one paint stations is at least one-fifth 
the number of cameras . 

17. The system of Claim 1, further comprising: 
at least one calibration station. 

18. The system of Claim 17, wherein said at least one 
calibration station is capable of creating a point 
calibration table for each of said plurality of 
cameras . 

19. The system of Claim 17, wherein said at least one 
calibration station is capable of creating a zoom 
calibration table for each of said plurality of 
cameras . 

20. The system of Claim 17, wherein said at least one 
calibration station is capable of creating a focal 
calibration table for each of said plurality of 
cameras . 
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21. The system of Claim 17, wherein the number of 
said at least one calibration station is at least one- 
fifth the number of cameras, 

22. The system of Claim 1, further comprising: 
at least one video storage device. 

23. The system of Claim 22 , wherein said at least one 
video storage device is a plurality of digital disc 
recorders . 

24. The system of Claim 22, wherein said at least one 
video storage device is a file server. 

25. The system of Claim 23 , further comprising: 

a digital router connecting the outputs of each of 
said plurality of digital disc recorders; and 

a first slow motion controller. 

26. The system of Claim 25, wherein said slow motion 
controller is capable of selecting a router output 
from the plurality of digital disc recorders. 

27. The system of Claim 25, wherein said slow motion 
controller is capable of controlling each of the 
plurality of digital disc recorders simultaneously. 

28. The system of Claim 27, wherein said slow motion 
controller is capable of controlling the forward and 
backward motion of the output of each of said 
plurality of digital disc recorders. 
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29. The system of Claim 25, further comprising: 

an additional digital disc recorder connected to the 
output of said digital router. 

30. The system of Claim 29, further comprising: 

a second slow motion controller for controlling the 
output of said additional digital disc recorder. 

31. The system of Claim 1, further comprising: 

a communications medium coupling the plurality of 
cameras to the master broadcaster computer. 

32. The system Of Claim 31, wherein said 
communications medium is fiber optic cable. 

33. The system of Claim 32, wherein said fiber optic 
cable is multi-mode fiber optic cable. 

34. The system of Claim 31, wherein said 
communications medium is triaxial cable. 

35. The system of Claim 34, wherein a semiconductor 
in said triafcial cable is used to modulate camera 
telemetry information and captured image data. 

36. The system of Claim 31, wherein said 
communications medium is a wireless RF connection. 

37. The system of Claim 1, further comprising: 
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a cam-A computer. 

38. The system of Claim 1, further comprising: 
a plurality of microphones; and 

a microphone computer for combining the outputs of 
said plurality of microphones. 

39. The system of Claim 38, wherein said microphones 
are directional microphones. 

40. The system of Claim 38, wherein said microphones 
are spaced around a target object that is being 
recorded. 

41. The system of Claim 38, wherein said computer is 
capable of overlaying the output from each of said 
plurality of microphones in the same moment of time 
based on the speed of sound and the distance from each 
of said microphones to a target object. 

42. The system of Claim 41, wherein the calculated 
speed of sound includes an adjustment for the altitude 
of the microphone and the relative humidity at the 
site of the microphone. 

43. The system of Claim 41, wherein the output of 
each of said microphones is connected to a digital 
mixer which is controlled by said microphone computer. 

44. A multiple camera video method, comprising the 
steps of: 
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using a master pan head to position a master camera; 

calculating telemetry settings for a plurality of 
slave cameras based on the master camera telemetry and 
a geometric transform in a computer remote from said 
plurality of slave cameras; and 

communicating the calculated telemetry settings to 
said plurality of slave cameras. 

45. The method of Claim 44, wherein said master pan 
head positions said master camera from a remote 
location. 

46. The method of Claim 45, wherein said master pan 
head and said master camera communicate via an 
Ethernet connection. 

47. The method of Claim 45 , wherein said master pan 
head includes a monitor that displays the video feed 
captured by the master camera. 

48. The method of Claim 44, wherein said master 
camera and said plurality of slave cameras each 
include a robotic pan head for positioning the 
cameras . 

49. The method of Claim 45, further comprising the 
step of: 

storing the video feed from said master camera and 
said plurality of slave cameras in a storage device. 
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50. The method of Claim 49, wherein said storage 
device is a plurality of digital disc recorders . 

51. The method of Claim 49, wherein said storage 
device is a file server. 

52. The method of Claim 49, further comprising the 
♦steps of: 

capturing sound from a plurality of locations around a 
target object; and 

adjusting the timing of the captured sound from each 
of the plurality of locations to compensate for the 
effect of the local relative humidity on the speed of 
sound so that a target sound is in phase from each 
location. 

53. The method of Claim 52, wherein the compensation 
for the effect of the local relative humidity also 
includes a compensation for the altitude of each of 
the plurality of locations. 

54. The method of Claim 49, further comprising the 
step of: 

producing a replay video feed based on the stored 
video feeds from said master camera and plurality of 
slave cameras. 

55. The method of Claim 54, wherein said production 
step includes the steps of: 

using a slow motion controller to select from the 
video feeds captured by the master camera and the 
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plurality of slave cameras as a current output source; 
and 

using said slow motion controller to move forward or 
back ward through the current output source. 



56. The method of Claim 55, further comprising the 
step of: 

using the slow motion controller to select a second 
feed captured by the master camera and the plurality 
of slave cameras as a second output source after said 
current output source. 



57. The method of Claim 54, wherein said produced 
video feed is recorded to a digital disc recorder. 

58. The method of Claim 57, wherein the produced 
video feed recorded on said digital disc recorder is 
further recorded on a second digital disc recorder 
attached to a second slow motion controller. 



59. A method for calibrating a multiple camera video 
system, comprising the steps of: 

capturing a plurality of calibration point values as 
absolute coordinates in three dimensional space; 
storing said plurality of absolute coordinates in a 
data file; 

capturing pan and tilt settings for a plurality of 
cameras when said camera is aimed at each of said 
calibration points; and 

creating a software geometric transform relating the 
three dimensional absolute coordinates to pan and tilt 
settings for each of said plurality of cameras. 
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60. The method of Claim 59, wherein said calibration 
point absolute coordinates are captured using a 
surveying theodolite. 



61. The method of Claim 59, wherein said calibration 
point absolute coordinates are captured using a global 
positioning device. 



62. The method of Claim 60, wherein said theodolite 
is adapted to .automatically download said absolute 
coordinates to a data file in a computer. 

63. The method of Claim 59, wherein said pan and tilt 
settings are read from a master pan head that is 
remotely controlling said cameras. 



64. The method of Claim 59, wherein said captured pan 
and tilt settings are downloaded into a second data 
file. 

65. The method of Claim. 64, further comprising the 
step of: 

generating a zoom calibration table by associating a 
field of view with a plurality of predefined zoom 
settings for each of said plurality of cameras. 

66. The method of Claim 65, wherein said zoom 
calibration table is in the second data file. 



72 



WO 03/041411 



PCTYUS01/46357 



67. The method of Claim 65, further comprising the 
step of: 

generating a focal calibration table by associating a 
camera focus setting with each of a plurality of 
selected focal distances for each of said plurality of 
cameras . 

68. The method of Claim 67, wherein said focal 
calibration table is in the second data file. 

69. A multiple camera video system, comprising: 
a plurality of cameras; 

a master pan head for positioning a selected master 
camera from said plurality of cameras; 

a master broadcaster computer for calculating 
telemetry for at least one slave camera from said 
plurality of cameras; and 

at least one paint station connected to said master 
broadcaster computer. 

70. A multiple camera video system, comprising: 
a plurality of cameras; 

a master pan head for positioning a selected master 
camera from said plurality of cameras; 

a master broadcaster computer for calculating 
telemetry for at least one slave camera from said 
plurality of cameras; and 

at least one calibration station. 
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71. The system of Claim 70, wherein said master pan 
head and said calibration station axe remote from said 
plurality of cameras. 

72. The system of Claim 70, further comprising: 
a plurality of calibration stations. 

73. The system of Claim 70, further comprising: 
at least one paint station. 

74. A multiple camera video system, comprising: 
a plurality of cameras; 

a master pan head for positioning a selected master 
camera from said plurality of cameras; 

a master broadcaster computer for calculating 
telemetry for at least one slave camera from said 
plurality of cameras; and 

at least one video storage device is a plurality of 
digital disc recorders. 

75. The system of Claim 74, further comprising: 
at least one calibration station. 

76. The system of Claim 75, further comprising: 
at least one paint station. 
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77. A multiple camera video system, comprising: 
a plurality of cameras; 

a master positioning device for positioning a selected 
master camera from said plurality of cameras; and 

a master broadcaster computer for calculating 
telemetry for at least one slave camera from said 
plurality of cameras. 

78. The system of Claim 77, wherein said master 
positioning device is located remote from said 
plurality of cameras. 
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Data Point Number 


Dimension 1 


Dimension_2 


Dimension 3 
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2 
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CAMERA PARAMETERS FILE FOR CAMERA NUMBER 1 

ID number 01 

Initial pan, tilt -18.078802 26.650000 
Calibration information 
Calibrated 1 



Location -40.273 


one ncn 
-305.950 


99.497 












;Four Comers 
















0 0.0 


0.000 


0.000 


0.000 


0.000000 


0.000000 


000 


000 


1 0 0 


0.000 


0.000 


0.000 


0.000000 


0.000000 


000 


000 


2 0 0 


0.000 


0.000 


0.000 


0.000000 


0.000000 


000 


000 


3 0 0 


0.000 


0.000 


0.000 


0.000000 


0.000000 


000 


000 


Number of GRID 


points 51 














;GRID Points 














0 1 1 


0.000 


0.000 


0.000 


0.650000 


24.840000 


255 


226 


1 1 1 


0.000 


0.000 


0.000 


-1.362334 


24.106688 


255 


226 


2 1 1 
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0.000 
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-3.230000 
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3 1 1 


0.000 
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-3.790000 


23.149117 


255 


226 


4 1 1 


0.000 
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255 


226 


5 1 1 


0.000 


0.000 


0.000 


-4.270000 


25.040001 


255 


226 



;Range vs. Focus Table (inverse range vs focus param) 

Number Entries 9 

0.000000000000 227 

0.003005353501 227 

0.003266036045 226 

0.003510695882 226 

0.003540278412 226 

0.003738491796 226 

0.003873184789 226 

0.004175557755 226 

;Zoom vs. Field of View Table 

Number Entries 256 

000 49.740 

001 49.740 

002 49.740 

003 49.740 

004 49.740 

005 49.740 

IUMUWUUMHJU 

250 3.020 

251 3.020 

252 3.020 

253 3.020 

254 3.020 

255 3.020 

Zoom Table Corrections 
Max_Zoom_Value - 230 
Min_Zoom2value - 30 
Min_Zoom_Useable - 31 
Max_Zoom_Value_Fov - 2.080 
Min_Zoom_Value_Fov - 34.000 
Min_Zoom_Useable_Fov -7.480 
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FIGURE 15 
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FIGURE 17 



402 



402 



402 



402 



H I 



400 



402 



I. 



402 



r 



402 



r 



402 



404 




SUBSTITUTE SHEET (RULE 26) 



WO 03/041411 



PCT/USO 1/46357 



20/20 

FIGURE 18 








410 



SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



Interracial Application No 

PCT/US 01/46357 



f P rf n ToTNr/T8 BJECTM «5/232 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 H04N 



Documentation searched other than minimum documentation to the extentthat such documents are Included In the fields searched 



Electronic data base consulted during the International search (name of data base and, where practical, search terms used) 

EPO-Internal > PAJ, WPI Data 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category • Citation of document, with Indication, where appropriate, of the relevant passages 



Relevant to claim No. 



EP 0 529 317 A (SENSORMATIC ELECTRONICS 
CORP) 3 March 1993 (1993-03-03) 
abstract; figures 3-6 
column 6, line 10 -column 6, line 39 



EP 0 714 081 A (SENSORMATIC ELECTRONICS 
CORP) 29 May 1996 (1996-05-29) 
column 3, line 6-38 -column 4, line 20; 
claim 2; example 6 



US 5 699 444 A (PALM CHARLES S) 
16 December 1997 (1997-12-16) 
abstract; figures 3-6 



1,44 



2-43, 

45-58, 

69-78 

1,44 

2-43, 

45-58, 

69-78 

1-58, 
69-78 



m 



Further documents are listed In the continuation of box C. 



0 



Patent family members are fisted in annex 



• Special categories of cited documents : 

"A*, document defining the general state of the art which is not 

considered to be of particular relevance 
"E* earner document but published on or after the International 

filing date 

V document which may throw doubts on priority daim(s) or 
which Is cited to establish the publication date of another 
citation or other special reason (as specified} 

"O" document referring to an oral disclosure, use, exhibition or 
other means 

■P" document published prior to the international filing date but 
later than the priority date claimed 



T later document published after the International tiling date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory underlying the 
invention 

"X" document of particular relevance; the claimed Invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

"Y* document of particular relevance; the claimed invention 

cannot be considered to involve an Inventive step when the 
document is combined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 

document member of the same patent family 



Date of the actual completion of the international search 



2 July 2002 



Date of maJBng of the international search report 



12/09/2002 



Name and mailing address of the ISA 

European Patent Office, P.B. 581 8 Patentiaan 2 

NL«228QHVRUswIJk 

TeL (+31-70) 340-2040, Tx. 31 651 epo nl, 

Fax: (+31-70) 340301 6 



Authorized officer 



Brandenburg, J 



Form PCT7ISA/210 (second sheoj) (July 1992) 



page 1 of 2 



J 



INTERNATIONAL SEARCH REPORT 


"iiitm^Phal Application No 

PCT/US 01/46357 


C.(Conttnualion) DOCUMENTS CONSIDERED TO BE RELEVANT 


•Category 0 
» 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


X 


WO 00 08856 A (KONINKL PHILIPS ELECTRONICS 
NV) 17 February 2000 (2000-02-17) 
page 1, line 3-26 -page 9, line 18-32 




1,44,69, 
70,74,77 


A 

* 


WO 01 08417 A (BOK JOSEPH CHARLES) 
1 February 2001 (2001-02-01) 
abstract; figures 1,2 
page 12, line 7 -page 13, line 7 




1-58, 
69-78 



Form PCT71SW210 (cortbiuallon d second sheet) (July 1992) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 



InHfhational application No. 

PCT/US 01/46357 



Box I Observations where certain claims were found unsearchable (Continuation of item 1 of first sheet) 
This International Search Report has not been established in respect of certain claims under Article 17(2){a) for the following reasons: 
1 I Claims Nos.: 

1 — 1 because they relate to subject matter not required to be searched by this Authority, namely: 



2 Claims Nos.: 

— because they relate to parts of the International Application that do not comply with the prescribed requirements to such 
an extent that no meaningful International Search can be carried out, specifically : 



3 ' because they are dependent claims and are not drafted in accordance with the second and third sentences of Rule 6.4(a). 

Box II Observations where unity of invention is lacking (Continuation of item 2 of first sheet) 
This International Searching Authority found multiple inventions in this International application, as follows: 

see additional sheet 



1 . f" I As ail required additional search fees were timely paid by the applicant, this International Search Report covers all 
I — I searchable claims. 

2. [~] As all searchable claims could be searched without effort justifying an additional fee, this Authority did not Invite payment 

of any additional fee. 



3. I I As only some of the required additional search fees were timely paid by the applicant, this International Search Report 
I — I covers only those claims for which fees were paid, specifically claims Nos.: 



4. [y~| No required additional search fees were timely paid by the applicant Consequently, this International Search Report is 
restricted to the Invention first mentioned in the claims; rt Is covered by claims Nos.: 

1-58, 69-78 

Remark on Protest Q The additional search fees were accompanied by the applicant's protest. 

| | No protest accompanied the payment of additional search fees. 



Form PC77ISA/210 (continuation of first sheet (1)) (July1998) 



International Application No. PCT/US 01/46357 



FURTHER INFORMATION CONTINUED FROM PCT/ISA/ 210 



This International Searching Authority found multiple (groups of) 
inventions in this international application, as follows: 

1. Claims: 1-58, 69-78 

Multi camera video system with master camera and slave 
cameras following the master camera via geometric transform 
and telemetry data 



2. Claims: 59-68 

Method of calibrating a multi camera video system using 
calibration points 



m INTERNATIONAL SEARCH REPORT 

fJ mtormallon on patem family memliers 


Intertl^ial Application No 

PCT/US 01/46357 


ftf$§2?* Patent document 
cited in search report 


Publication 
date 


Patent family 
members) 


Publication 
date 



EP 0529317 



03-03-1993 



US 
BR 
CA 
DE 
DE 
EP 
OP 
OP 



9203284 A 

2064299 Al 
69216005 Dl 
69216005 T2 

0529317 Al 

3268019 

5207467 



B2 
A 



06-04-1993 
23-02-1993 
30-01-1997 
03-07-1997 
03-03-1993 
25-03-2002 
13-08-1993 



EP O714081 



29-05-1996 



AU 
Al) 
BR 
CA 
DE 
DE 
EP 
OP 



701222 B2 
3798595 A 
9505248 A 
2155719 Al 
69521479 Dl 
69521479 T2 
0714081 Al 
8265741 A 



21-01-1999 
30-05-1996 
16-09-1997 
23-05-1996 
02-08-2001 
11-10-2001 
29-05-1996 
11-10-1996 



'fir 



US 5699444 A 16-12-1997 NONE 



WO 0008856 


A 


17-02-2000 


US 


6359647 Bl 


19-03-2002 






WO 


0008856 Al 


17-02-2000 








EP 


1042917 Al 


11-10-2000 








OP 


2002522980 T 


23-07-2002 


WO 0108417 


A 


01-02-2001 


AU 


6890400 A 


13-02-2001 






WO 


0108417 Al 


01-02-2001 



Form PGT/ISA/210 (patent family annex} (July 1892) 



